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SUMMARY 

During the period 1974-1978 a research Jjrogramme^ was to in-^ 

vestigate alternati^^^^ Ordinary grade English: The purpose 

was to investigate the validity, reliability, practicality, and genAral deslriabHity 
of means of asses^^^^^ soibstantial element of school judgment: 

The experiment involved comparison, of the results of assessment of English 
candidates by three methods: {i) the Ordinary grade examination, (it) a system 
pfiJolip assessment' and (iii) a 'criterion test' designed by the SCRE research 
t^m tagive a comprehensive, valid arid reliable assessment of the objectives 
of the Ordiriaiy gradb English course. Of the fourteen schools taking part ten 
provided full data for the experiment arid brie other provided, partial data. 

The fdlldwirig gerieral coriclusibris may be drawn from the research: 

The Ordiriaiy grade Ejigli^^ has many advantages, its admini- 

strative arrangements are weii-tried and effective, and its riatibriwide eurr^enciV 
aisures comparability of staridardj for j)upiU: fr^^ all schools; in this respect 
th_e_re is particular advantage in having a^single common interpretatibri_test. 
There is a general satisfactibri with the OrUiriaiy grade examination among 
bbtli teac^fwrs and oth users of its results; A further credit to_the examinatibri 
IS the opportunity it affords to some candidates to show their real worth, which 
they may hayejacked the motivation to reveal in school work; this benefit is, 
of course, offset by the disadvantage to pupils whb wbrk well in school but 
are 'bad exairiiriees'. 

J^P^J^ore un^^ of the Ordinaiy grade examination, 

besides any undesirable influence on teaching, are threefbld: 

(a) it is bound to be of limited validity because it can sariiple only a small 
prbportibri of possible English work; 

(b) like all assessment of English, it suffers, despite marker-standardisatibri, 
from serious inter-markef iricbrisistericy; 

(c) it is ilbt a firie discririiiria/or. 

There are rip pbyious means oV reducing the effects of these major disadvaii- 
tages, short of the probably impracticable solutibn bf riiultiple- or, at least, 
double-nlarkirig to iriiprove riiarker-reliability. 

: J/on the other hand asses^me^^^^^ at the end of S4 were whoiiy 

internal, the main advantages would be gains in validity. Teachers would pro: 
bably see educatibniil arid professioria] advantages in internal assessment, and 
\ypuld be likely to feel more free in practice to teach a variety of cburses arid 
would plan them mqre_ caferully. Iri-service trairiirig through Trial Marking 
exercises would be needed. 

Numerous prbbrem arise. While many schools, with ap- 

propriate in-service training in assessment, would assess their pupils satisfactbrily, 
a number would fail tb do sb fbr a variety of reasons. The setting of valid and 
C:Oiriparai>le interpretation t^ or in different schools presents 

great -difficulties, which affect both schbbl exariliriatibris and Folio assessment, 
especi^yiy^heiattAr.::M^ would be at least as: prevalent as in 

the Ordinary grade examination, and the schools wbuld be slightly mpne likely 

'Zurich' the marks th^ grade markers, so that they would prove 

no better discriminators: In addition, there would be sbrile dariger of schools' 
^n^^^'^'^arking uricbbperatiy^ some thought to be of 

high quality because they are in top classes. Moderation presents practical arid 
theoretical problems. Above all, the difficiilties for teachers of implementing an 
iriterrial assessment scheme in current working conditions are considerable. 
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A combination of [nternal and cxtehtal marks for jHesc areas <)f work would 
in JTiost eases be a better assessment than either alone, provided that both are 
validly assessing some aspects of English. It am be shown that the reliablHty 
discriJTiinati^^^^ Improve when difterent 

single-marked assessments are added together, although the imprbvement is riot 
so X"arked as when jTiuItiple-ma^^ out. Even 

when internal assessment was significahtiy Jess satisfactory than external in the 
SCRE project, the average of the two correlated satisfactorily with the Criterion 
measure. 



lA <:pj7ibined assessment scheme would, in principle, allow a reduction in the 
size of the external examination, sirtcc the writirig task could be shbrterjaileast 
when eertain types of task were set) and it would be possible to leave the assess- 
ment of literature wholly to the schools,* if one were prepared to accept the 
•J^sadvantMeJHat necessary to scale internal writing and literature 

against external writing and interpretation marks. Depcrtdirtg bri success iri 
^^X^l9P^"S them an^Ltheir acceptance by English teachers, multiple-choice tests 
of interpretation might be employed^ In any case, there may be a need for the 
asscssn\ent of reading iri schbols tagi^ literature 
and to kinds of reading other than passages similar to those used in the Ordinary 
grade examination.' 

The priicticul problems of internal asscssrtlerit would still apply to a combined 
scheme^ but would be more manageable without the need to set interpretation 
tests on the O-grade pattern or to prepare pupils for ari external literature test. 
The researchers^ believe that the professional advantages to teachers in the ac- 
quisition of knowledge aboat, and practice in, assessment, albrig with the im- 
P'^bveriierit iri vaMity arid reliability such d system would probably bring, out- 
weigh the difficulties it would meet: Nevertheless, it would be advisable tb 
reduce the bUrderi bf assessment carried by iteachei s by req uiririg internal 
assessment of about ten pieces of work, rather than the fourteen or fifteen 
demanded by the SCRE project. 

- Inciden tally to the rilairi thf us t bf its wbrk» the project has brought to ligh t 
in the Scottish context the same difficulties in defining achievement in ike subject 
and in isuring comparability of Judgment ambrig asscssbrs as bcciir Jii all 
syst^^^s o/ assessme^ in English. The researchers suggest that there might he 
benefits to the teaching of English If teachers were more aware bf these difficulties 
^1^^ 1^ tlj^ey shifted the assessment from assessment for 

di5cri:;:ination to assessment to describe pupils'—strengths and weakriesses iri 
seellir:0 to achieve the puipbses th^ work., The amount of teachers' very 
limited assessment time to be devohd to the two types of assessment depends 
ultima^:;ly on value judgments abbut the relative iriiportarice of each. 
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INTRODUCTION \ 

Aims , 

The -SCRE investigation nor ^^^^^^^^^ assessing Ordinary grade 

Englisji for the SCE Examination Board began in September 1974 with the 
following terms of reference quoted from the proposal Tor the research pro- 
gramme as finally revised in March 1974. 

'Aims _ : - ' 

1. To report on the present practice in assessment by schools of candidates 
for Ordinary grade English. 

2. To investigate and determine the optimum ways in which teachers can 
make assessments (possibly in the fortri of brdersipimeritj of pupil per- 
rorm£Hice_in selected aspects of English^ these assessments being based on 
a wi^de range of work over a substantial portion of the school year. 

3: To dctemiine the best ways of scaling^^^ these assessments 

so that assessrrients of pupils in different schools may be comparable. 

4. To investigate such other related matters as Jiiay appear in the course of 
the investigation to be relevant to the field of interest covered fc vnc 
Board's remit. 

5. On the completion of the above steps^ to make recbmmeridatioris ta the 
Board on whether alternative meansiiutilising teachers' assessments, should 
be substituted for the whoje ox part of the present O-grade examiriatibri 

E^psh, to outline the adm would be involved. 

Costing of these changes might also be undertaken in cooperatibn with 
officials of the Board. 

Objecti>£s 

^- fe) 1^ Cete^rm by rneans of a survey^ the methods at present used by 
presenting centres in thepreparation of Order of Merit Hsts (Form Ex. 4j 
for candidates jjreiente^^^^^ grade English. (The survey should 

determine the proportion of such lists which are based on brie, bri twb, 
on three, arid bri four br riibre internal assessments of the candidate.1 
{^) cletermine the validityiof such Order of Merit lists as measured by 
the SGE examination: (Should the results of 1:(^) allow, separate nieasures 
yaH^^^^^^ shbuld be deter^^ for each of the categories of internal 
assessment procedures identified:) 

2. (a) To note the objectives, as stated by SCEEB. of the Ordinary Jrade 
assessment bf En^^^^ as at present constituted, and to consider whether 
any further objectives have been implied in the O-grade papers set in the 
last Jew years. 

(b> To list, after cph_sultatio-i with practising teachers of English, other 
objectives, if any, of the study of English in secondary schbbls which are 
not eiirreiitly assessed. 

3. Tp_ isolate apprbpriate objectives as defined jn 2 for which the research 
programme will investigate the feasibility of internal assessrilent by schools. 

4: To establish, /^?r the pur pq^^^^ 

eedure that would be used to evaluate as comprehensively asipossible all 
aspejcts of tlie performance of pupils studyirig^^riglish iris Secondary 4 
Sreparato^ for presentation for 0-grad<\ (This procedure would probably 
he too time-consuming to be employed other than for experimental 
^purpbses.) 

5. To devise one or more practicable systems of Intemai assessment judged 
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to meet the objectives stated in 3 and employing where appropriate some 
yOf the techniques liscd in the assessment procedures described in 4. 

6: To devise a practicable monitoring system for the internal assessment 
systems described in 5 and to r<$drt oin the yalidity such monitored 
teav^hers' assessments as measured by comparison with: (a) the relevant 
aspects of the SCE O-grade examination, and (b) the assessment system 
deviled under 4 above.' 

This statement of aims and objectives was interpreted by the research team^ 
undcr the influence of their- owr^ educational experience and concerns and those 
pEihe yarigiil members ofi i^oniMiltee, to im^ that Htheir work 

had two broad purposes, one with an assessment emphasis and one with a 
*cu r ri eu 1 u m id ey e 1 opmcnt ' slan t. Th e 'aj teirl at i ve mea s^ o f lassess i ng O-grad e 
English were taken to be ^in-school' assessment and the project did not dircctly 
con sider pbssi ble changes in the external exam inatio th b ugh the Research 
Officer was co-opted on to a sub-committee of the SCEEB English Panel which, 
over two years, was developing, pre-testing and Judging the value of Multiple- 
choice tests of interpretation as possible comporients of future examinatiotis. As 
to its assessment emphasis, the project sought to provide infomation for SCEEB 
about possible forrns of internal ass^ iheir practicat feasibility: find their 

comparability with two other assessments: the present O^grade examination and 
the -cbmpf ehensiye assessHicnt pjoc^duj& designed specJaH^^ for the jjrcyeet. At 
the same time the educational advantages and disadvantages for pupils were to 
be <i_escribed and a report made of the irri^^ on the 

teachers' attitude to developing their own courses and of the actual effects, if 
any, of the greater freedom allowed by internal assessment to engage in a variety 
of English activities. Ft was recognised that the project was concerned with the 
evaluation of a form of assessment already in fU'rly wide use elsewhere, but it 
was thought desirable to describe its effects in the Scottish context. 

Procedures 

The procedures adopted can be summarised in three stages, occurring in 
roughly chronological order, though there was some overlap between them, 
since one was being prepared while anbthcr was in progress. 



1. A Fact-finding Stage ^ _ i_ 

W hat was actually i>ein^ d one in sciiools in 1 974^75 to pr6y[de the rank o rder 
of candidates sent by each school to SCEPB before the O-grade examination? 
What were teachers' views on the O^grad an assessment in- 

strument and as a m.cans of evaluating their courses? How did they react to 
the idea of internal assessment? What aspects of English work were notxpvered 
by the external examination arid which of these were teachers cpncemed with? 
Answers to these questions were found by survey methods, and an evuiuatJon 
of assessmerit rriethods in current usc was made by comparing schoo^^^ rank orders 
or marks with the 1975 O-grade results: This stage met objectives I and 2 in 
the proposal statement. 

i A Development Stage 

Ol^ectives 3. - 6. delineate the bulk of the work of the project, the experimental 
implementation of an in-schooi assessmejit system and the evaluation of it. The 
development stage, from Spring 1975 to Summer 1977, involved principally two 
undertakings: ^ 

(ii) the estabrishmeht of a thorough aridl reliable assessiiient prcKcdiirc to 
serve as an independent criterion with which to coriipare both school 
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Pri^^S ? -^^^^^ assessment came to be 

called The Criterion Test'; 

,{b) |he planning and implenientatiori of un internal assessment scheme and 
tne means of monitdritig it. 

3/ An £\^iaation Stage 

Judgments about the ad and disadvantages of the scheme as an assess- 

es ^f?^^ ^"^ ^ ^P"^ cufriculum planning have been made on the 

(a) comparisons between the three assessment measures is to rankinc and 
standards; 

(b) c^mmenls obtained a^^^ teachers during the year arid 
their responses to a questionnaire at the end of the experiment; 

(c) the practical problems which arose for the teachers and the researchers. 



\ 
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CHAPTER I 



FACT-FlNDlfS'G: Vi€WS AND PRACTieES, 1975 



The development work of the project, towards establishing an independent 
Criterion Test and towards an internal assessment scherhe, was carried'but with 
some awareness of teachers' views on S4 English and of assessment methods 
in current use in 1974-75. Three surveys were made during that academic year: 
one, covering 25 % of the schools presenting candidates for the SCE O-grade 
English examination, sought information about the means by which the schools 
drew up tftCiOrdef of Merit lis^ on SCEEB's form Ex. 4, and was 

followed up by a study of the accuracy of school assessments as predictors of 
Q-gtade periomance^the other t^o,:bet^weri the aifurthLe* 
5G% of d-grade schools, w^re xioncemed with the aims of English teaching 
preparatory to O-grade, teachers' the pre»ent:_exa^ ^hd Jheir 

reaction to the idea of internal assessrnent. The questionnaire method used in 
all three surveys w^ 

about current assessment methods than for discovering the truth about teachers' 
views and their teaching purposes. The process of Writing the questionnaires 
and the analysis of responses to them did, however, give the researchers a general 
impression of the state of SFV English teaching and assessment in 1975. 



I. Surveys of TeachersV Views 

The principal reason for drawing up the lists of 'objectives- and 'criteria' which 
formed the bulk of the two questionnaires was to map out for the planning of 
a Criterion Test the ^yhbIe field of S4 English work. The main content of the 
questionnaires is therefore discussed in Appendix 3, which describes the develop- 
ment of the Cri tenon Test. Copies of tjie questionnaiEes are printed in^ Appendix 
j. In both questionnaires the word 'objectives* ^eans ^skills earning credit in 
assessmeht' : the expian terni isigiven in the 

preliminary paragraphs of Appendix 3. Summaries are given here of those 
findings of the two surveys which seemed significant to the research team. 

{a) Free-response questTonhaire 

The Principal Teacher of English and one other (randomly chosen) teacher 
from his department in_107 jchoois were invited to consider, comment on 
and add: to a list of ass^^^ the teaching aims they imply, 

derived from analysis of SCE O-grade qu^tions and markers' instructions; 
Stry^es arid Ayor|:-tg-rure by tea^ aflfected ithe res^^ 

that only j f 5 teachers in 59 schools completed the questionnaire. Their 
cdmments carin^^^^ but a number 

of indications were given of some matters of concern. 

1. As to curriculum^ there was some evidence that a number of teachers 
would very much like a clear statement oT what they should be teaching, 
especially with regard to grammar, correct usage, spellirig arid purictu- 
ation. There was, however, a broad cross-section of opinion and many 
empha^s^d what they saw^sthe restTictip^^ 

on their freedom to design their own syJIabuses. Nineteen of the 115 
specifically referred to the Mlmitedriess^'^ the s^^^ the examination. 
The largest group of respofkdents in agreement with one another saw 
a need to develop and assess oral English (50/115), arid another group 
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(17/115) inciaded among their aims the development in pupils of a 
cntical mind^witfe re^ to the mass-media. _ j 

These poiats apart, there. was a general agreement that the SCR E 
analysis of O-grade. cFiteria and aims Fairly reflected what S4 classes 
were in fact working towards and that, on the whole, it had produced 
a statement of the appropriate sort of work for 16-year-oLds. 
^s tb epiaipnAab^ of assessment, coniraents positively ap- 

proving of some form of internal assessment (60/1 1 5} outnumbered a 
nevertheless fairly large group of responses which expressed positive 
: approval of the existing examination (22/115). Stfbilg disapproval of 
mtemal assessment, nz^ri the grounds that it would be impossible to 
standardise, was indicated by 5/it5 teachers. 

^^^J^^^^^ some suspicion o^f critena:^^^^ writing which referred 
bisido the Q-grade marking instructions) to 'personality', Torcefulness', 
andi^sincerity';- 20/1 15 teachers suggested that the a^^^ of these 

<iuayyes,:espec_ially ^t is not possible in an examination. A group 

of about the same^size (but not necessarily the sarne teachers) emphasised 
the impert^ce of 'correcthess' as a criterion of good English; 

An odd mixture of opinions em'erged about the Reading sectLari of 
the examination^ some the implied aims were too difficult, 

others that,^ in practice, the questions set are so 'open' that they dp hot 
^^^JJy, test the extent and MaKty of the pupil's reading ^t all. Since two 
examiners who had been cojisulted had been able to reconcile the *open' 
questions with the aims^nd eritcTia _s^^^^^^ perhaps 
there is a lack of-communication between the SCEEB and teachers as 
to what exactly the 'Reading' questions are trying to test. 

(bj ELxeches'po/isequestiajvmire - 
Mainly on the basis of in the free-response survey, a detailed 

list of - objectives' for writing, interpr^taSon and literature work was drawn 
up and sent to all English teachers iiv 10? schools (a 25% sample, stratified 
t>y Education authority arid by type of <?chool. viz: 6- Year Comprehensive, 
4- Year Comprehensive^ Selective). The teachers_were to indicate whether 
thpy ex53eK>ted each object! be achieved by pupiis obtaining Cpass in 
. the Q grade, or by pupils obtaining A pass. Some items ^WB-WSy were 
^^^^^ designed lo obtam^ to oral English, to 

internal assessment and to the present O-grade examination. : 

^j^'c hundred and twenty-three teachers in 94 schools returned completed 
questionnaires. This was a good response from schools (86%). It was riot 
possible to know exactly how many respdnseszrrpm individual teachers to 
expect, but, assuming an average of 9 teachers per school, the 523 probably 
constitute about 60 % of the total sarilple. Withm theJimUations of question- 
yaire data^it may be taken that the trends of the responses of these teachers 
represent roughly the 5tate of English teachers' thinking abjut O^rade 
^'^ prk. WhejFver^ss-^^ were made, no consistent pattern of responses 

was apparent in schools of the same type or among Prirtcipal Teachers as 
oppojed to bthe^^ THc re each: item can be seen in Appendix 1, 

where the questionnaire is printed: Here the main trends of the teachers' 
choices are sumrtlafisedv zz r i • ^ 

It Avill he hpted by anyone consulting Appendix I that^no attempt was 
: made to obtain teachers' views bri the degree of Jophisticatiori with which 
cacji skill should be employed or each objective achieved^ The definition 
of precise criteria of achievement is not possible, because performance partly 
deperids on factors not susceptible to specification, such as emotional 
maturity and intellectual development. It is necessary to rejy on the ex- 
perience of exariiiriers or moderators to discriminate between scripts with 
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and scnpts withbUt sbphisticatibri. This of 
EnglisH criteria and objectives in the questionnaire should be borne in 
mind in considering the value of its results, as should the usUal fesefVatibns 
about questionnaire data : the information obtained is about teachers' views 
of aims, etc:, wiien nskeci about tfiem in tins way, not necessarily about their 
real aims or pjactiee^^^^ 

The following points among the views Indicated seemed significant to the 
researchers: 

1. There was general agreement that most of the very full list should be 
d-grade objectives. - 

THe_ skills most frequently regarded as being outwith the range of 
0-grade were: 

writing: argumentation fite^^^ 019-020; 022) 
sophisticated story. writing (037-043) 
poetry wntirig|045-04^ ::: 
using emotive and figurative language (048-049) 
some aspects of /correctness', yij: ^ u of the sera-colbn 
and colon, of the dash and parentheses, spelling (056-062) 

RHADiNG: comment on the writers' techniques and skill (070; 072; 
082-085) -_: : . 

judging the validity of ideas in a textj(080) 
some aspects of grammatical knowledge, viz., moods, par- 
ticipleb., voicesj(lO2-105). ^ , 

On average (save-ior poetry writing) only^ about 20 % x)f teachers 
bmitted these iterns^ The 

of other items, which the majority had chosen, to justify the conclusion 
^^^*"e |s some (th o ugh riot much) d isagreem en t- among Erigli s h 
y teachers about the importance or practicality of different kinds of work 
\ far O-grade pupils. - 

- • NT'i ough rilos t objectives generally regarded, ias_ ^art of Q-grade 
work, there was a tendency for many teachers to differentiate fairly 
shart)ly between 'C pupils' arid 'A pupils' with respect tb certain ob- 
jectives (cf. items 016, 017, 019, 020; 046, 047, 049; 063; 070; et al). 
SomeNdifficult objectives were thbught by a nunlbef bf teachers tb be 
attairiabJe by their *A pupils'; e.g. item ()42, write a story with a moral; 
or 072, glKe reasons for the jiidgment that o|ie piece of writjng is better 
th ari ano t fier . Th i s clear differentia ti on between *C ach ie vemeri tV an d 
'A achievements* is not justified by the actual difference in quality 
l^^tweeii scripte^ a^ range 8 (:=C) arid range 5i=A) in the 

examination. Awareness that there is some spread of ability among 
pupils taking O^ra^^ teacher^ to urider^^^^ 

difficulty of someskills they expect of *A candidates'. There may well 
be many teachers who reg^d the O-grade examiriatibri as. an 'academic' 
one for which they afeo present *n oh -academic' pupils. The iqipression 
thatmany teachers set f^her standards than the examiners was, in fact, 
corifirriied 1^ the^ study ofthe comparability of school examination marks 
with O-grade marks (see Il^Jbejow). 

3: A majority thought the pr^^t examinatibn is effective br, at least, 
ys^efu! for the purposes Tdv whlbh it is mainly used. A sizable minority, 
however, disagreed. (Items r09-rK2.) 

4. There Avas a^neM"ly equal division of opinion about the desirability of 
moderated internal assessment (Item ikl5). It was notable that 58-3% . 
bf the teachers consujted practised coriUmious assessment in_S_l-3, but 
only 21-6% did so in S4-6 (Items 113-114). (It should be noted that 
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• *cdritiniwus^|ses not defined In the questionnaire and may 

> have been understood by some to mean 'periodic testing'.) 
Scventy-twd need for planned otal work, 

but most thought that any assessment of it should be informal (106-108). 

11. Survey of Methods of A^essment 

Tixe: results of this survey; and of the study of school examihatidns as predictor 
of O-grade pegormance in 1975 have been discussed in a separate report*. Very 
briefly, the findings were: 

^ ' ^"^^^^"^^5 there were jjrobabli^^n^^ In current 

use: all 104 schools surveyed used *mock' O-grade examinations, while 
course work had very little or no infl^ehc^^ arid, in any cVse. included 
quite a large proportion of P^st Paper work. 

2. In general, school 'mock' maSs appeared ta correlate quite well with 
O-grade marks ^ most cdrrejatioris were^jetween -70 and -80. The study 
did not, however, exsmine closely the particular characteristics of each 
school, nor those of the O-grade exariiiriatLGn::itseI6 which served as the 
cTkeripnl measure. It was assumed that correlations of -70 or better 
indicated reasonably good prediction by the schdbUbf thejO-grade rank 
order. In latt^thou^ of a school's correlation would have 

been influenced, not only by the validity and reliability^ of the school 
exajninatidn, bU^^^ pF apility and number of pupils 

taking the two examinations (factors which^varied from school to school) ^ 
t^^Ayalityof th^ 

of a good deal of inaccurate prediction in certain schools, and in certain 
^^^^s w^^'^^" schools dbtainirij 'go^^ but at this stage 

|here>yas no further exploration of these indications: Thejnost obvious 
problem was judgment of the *pass' staridai^:-^ the C/D borderline. 
Most sc^hpols were^ severe than the examiners m placing this: line 
: on their rank orders <^ few were too lenient). Hdwever, the lack of 
"^atch |3etweeii 5chpo! examination ranking and - O-grade janking for 
average pupils, even in schools obtaining correlations df -70 upwards, 
was clear frdni the fact tha^^ 12% - 15%ofa * 

schbdl's. candidates to have been 'misplaced' across the C/D bdrderline 
i:e:, predicted td fail but actually passe^d, arid vice versa - even after 
ic^ing orthe schoors marks to adjust its C/D borderline to as near as 
possible to the O-grade standard. Several schddls 'riiisplaced' more than 
20% of their pupils. 

III. Ihterhal i^ssessnient Elsewhere 

^"^^"fi '^act-firidirig^ stage of yiej)r^^ to examination 

boards and schools In England which were implementing internal assessment 
in English, and infoimatidri was alsd dbtairied abp^^ : 
and Australian experience of the change from an extemai to an internal system. 
The fruits of this investigation have been repdrted iri detail in 'Iriterrial Assess- 
meriJ^Dr^Externa! EMminat^^ (Teaching English, January 1975). In summary, 
the perceived advantages of internal assessment were fdr teachirij;^ course plan- 
ning M^JugLlmptivMoniB this last); the 
major problenis of comparability and reliability and the costliness df riidderatirig 
systems constituted the rriam was a noticeable difference 
in commitment to internal assessment between the CSE boards' representatives 

« *Metfiods of assessment used in making order of merit lists for the 1975_b-grade_Engiish 
examiaation arid their aeeiiracy compared with the examination.' (Unpublished report to 
SCEEB, E. Spencer, 1975.) . 
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Ariose Prthe:^^ Board: the rormer tended to stress 

the advantages, the latter the disadvantages. It was, indeed, clear that proof of 
J"P^r»o^^ty of one or other method of assMsment was not available. Prefer- 
ence for internal or external methods was at least partly based on value-judgments 
•^^^^ P^n^oses of assessment and about its rpl^ in jhe pupils' education. 
There may also have been some Irrational failure to appreciate the faults of 
one's *own* system, or the merits of the 'opposing' one. 

IV. Suihinary of Fact-Finding Stage 

The English teaching world in which internal assessment vsajis to be tried out 
wM_^Pmmatjedt:avfa^^ concerned, by the 0-grade examination: 

Methods of assessment in current use and much of the schools' course work 
^"^'er^ n^pt^ejle^ direetjy on the O-grade papers. There was a general satisfaction 
with the examination, despite some misgivings about the 'Reading' questions 
A^ie w^^^ some lof the: o^^ employed in assessing 

writing; some opinions hostile to the examination were expressed, however, 
becau^ jts Mimiting' effect on the curHeu^^ desirability of 

mtenial assessment were evenly balanced - more so than support for the present 
examination might have indicated, because some teachers^ though sa_tisfi_ed wfth 
th(^ O^rade for some purposes, felt that internal assessment would offer new 
opportunities. 

it seemed that, an practice, most teachers were happy to let the examination 
determme-jheir:^ Indeed, it could be argued that the O-grade 

examination does cover a very wide range of English skills, if one takes account 
of ^11 possible opt^^^ the paper over a number of years and all possible 

criteria of merit employed by the examiners. (The achievements of no indjvidual 
P"jy| A*"^' 'lowever, ass^^^^ may have been this 

'openness' of the examination which led^ome teachers to regret the absence of 
V a clear, centrally imposed syllabus for S4 English work. Meahw^^ 
their colleagues, with apparent perversity, were objecting to the lack of oppor- 
tunity for them to develop their own curricula, due to the restrietirig influence 
Ptthe P-grade - inip^^ oral work, close reading of literature, certain 

types of writing and active drama were said to be ijnder-develbped. In con- 
sidering the work tji^ teachers tended towards rather 
idealistic expectations of the performance of 16-year-old pupi! 5, especially those 
^hdm they predicted to obtain ani A pa^s^ Yet^ agam were 
suggestions that less abJe pupils were not being^tretched enough by the reading 
material presented to them in preparation for O-grade. 
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THE INTERNAL ASSESSMENT SCHEME 
I. ' Number and Choice of Schools 

The size of the sample of schools the researchers co u Id invite to iparticip^^^ jii 
the project was Iiit)lteci by the amount of marking of the Criterion Test which 
could be completed during M 

markers there was money to pay^The estimated maxinium marking load was 
about 2,500^ scripts fc^ 

schools might be asked to assess the work of only a proportion of the S4 pupils 
in each \va^j*eiected, since it was thought that little infonnation would e^ 
from such an exercise about the problems for schools of internal assessment in a 
realistic context. If whole schools were in question, the research team could 
then choose at most 15 for the experiment. Such a sriall group of ischpplsi it 
was realised, might noi allow legitimate generalisations to be made from results, 
but against that misgiving could be set the following cbhsideratibhs: 

I. there would be gains from a whole school approach to practical problems; 

2: closer contact could be kept with the teachers; 

3. it was desirable jo have fav'ourably disposed teachers ^ince they were being 
asked, hot to replace their O-grade preparation with a different scheme, 
but to carry an extra burden; 

4. the purely administrative difficulties of dealing^with many more schools 
would tkke up research time; 

5: some valid judgments about national internal assessment could be made 
wi th ea u tiori^ if there _wer e n b reason s fo r beli eying that the 1 5 schools 
were in any way untypical of Scottish comprehensive schools; difficulties 
arising in some or all of 15 schools cbUld be expected to occur in others, 
and success or failjire by soine woiild point to the possibility of success 
or failure elsewhere. 

No attempt was therefore rhade Jo ohoo^^ 
*cominon sense' criteria were appliedito achieve a reasonable balance. Schools 
of yar^^ing^Jzes: :^A^^e rc^ in jdifferiint^eo^aphi^^ town 
and rural catchment areas; and including bp^btnon-denominational and Roman 
Cath q lie schoo Is. Th^ Steering Cqitimjttee^SfT^ prpjec t had suggested that at 
ieast one of the chosen schools would be one likely- to have difficulty in fulfilling 
the requirements of the experiment (in the event, several schools found them- 
selves in this condition); otherwise, schools could be selected on the grounds 
that fairly enthusiastic co-operation might be expected, though not necessarily 
100% commitment from all rriembers^ staff. Chie other qualific^^ 
manded by the Trial Marking* sclieme proposed iis part of the experiment: 
the selected scHQpIJ had to be wit^^ travelling distance of one another, 

to facilitate meetings of their representatives. 

Strathclyde, Fife and the iSranqjian Region were chosen as areas including 
a wide variety of schools, and letters were sent to Head teacher* describing the 
proposals and inviting participation. In Fife and Straticlyde the choice of 
schools depended mainly on the Research Officer's knowledge that the Principal 
Teacher or the Headteacher was likely to be interested, this knowledge, in spme 
cases^ having been passed on by the region's Adviser in Eiigjish o^ a: member 
of the Steering Comm^ 

cities were to be represented by the Glasgow conurbation, the Grampian schools 
y^re chpsenlmainly on the ^^asis^of their prox Replacemerits 
for the several sch. As which declined the offer were selected on similar grounds. 
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By April, 1976, 14 schoois constituted the experimental groups and were 
designated as follows: - 

Fife =^ Area i,iive schools _ \ 

Schools 11, 12, 13, 14, 15: \ 

Grampian = Area 2, four schools \ 
Schools 21, 22, 23, 24. V 

Strathclyde == Area 3, live schools \ 
Schools 31, 32, 33, 34, 35. I 

A sixth school in the StrathcJid groups the o^ cjipsen as Hke^^ 
difficulties due to staffing problems - withdrew at too sfiort notice for a willing 
replacement to be fouiid, though an attempt was made at the last minute to 
persuade another Grampian school to join in. \ 

jThiS:sel(W?tioTV of :14:Schools did in Tact, albeit thinly, cover most of the vari- 
ables one woiild seek to account for Jn a randqm^ sampld They ranged in size 
abmtt SGO pupilsJp abb^^^ were in 

large towns, fonr in small towns with some rural intake, :^d one was a village 
school with a largely rural eatelment areaj three wef e Roman CathpKc i and 
two werc^single sex (one boys% one girls'). It was found in due course that there 
werejn the 14 schools quite large variations in the ability range J>pth of the 
pupils presented for O-grade and of the S4 group as awitoJe. Finally, it was 
also later discovered that teacher-pUpil ratios in the English departments of all 
the schools were similar (except for schools 31 and 33, which had special tem- 
porary staffing difficulties at the time of the experiment). \ 

The pHricipdl types not jiccounted for at all in the group \were independent 
and selective schools. \ 

IL Type of Assessment \ 

It niight have been possible to engage the schools in any dfUeveral sorts of 
assessment procedure. The three most obvious possibilities were: (1) school 
examinations, (2) course work assessment (in which the pupils^ stored work is 
assessed atiithe en^^ cpurse Qt, sometimes^ also at brie\ or two points 

daring it) and (3) continuous: assessment, a method involving! the systeniatic 
polI^Ltipri oh course work or on special tests, 

and their aggregation into a finai grade. : 

While it can be argued that all three of these procedures can have educational 
value, that con^Jnuous asses aUow ino of deyi^tqpmehiy^^ 

that it is possible to define criteria of achievement for the award of various 
grades, a basic -concern of them alHs discrtmmattoh, and they aH rfely ultimately 
on itjie prgfessip?^^^^ effect this. The res&rchers were 

aware that,::if internal assessment is to play- a significant part injeducation, a 
sWft in emphasis is^j^ 

the world outside school and on to that (limited) area of pupils' |ives in which 
tlie Prdfessip best available_^ tp pupils- 

achievements, namely, academic performance and social behaviour within the 
school cohlext. Teachers are probably good at judging pupils' achievements in 
the pupil-rple within that context, and there is an important place for assessment 
schemes which facilitate description of pupils' achievements, definitiqti of skills 
and knowledge shown, and the pooling of several teachers' judgments about 
pupils' schooiwork, The SCRE publication, Popifs tn Frofite (SCRE. 1977), 
describes such a scheme. v ^ 

Assessment q^^ be cpmplemraje^^ 

cern to encourage^:^ self-evaluation by the pupils as the basis for discussion be- 
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tween pupil and guidance teacher about prdjress, moti^^^ relations with 
teaciiers, careers^ gui<la^ development, and so on: the researchers 

were impressed by the development of this aspect of infernal assessment at the 
Bosworth CoUege, Leicei^te^^^ felt that profiles and self-assessment 

ought to be among the concerns of aity study of internal assessmOTt^ibecause it 
is-in^these types of assessment that- significant would be J50th 

yaluaHe and possible. The fact is, however, that, as yet, the shm iiv emphasis 
aw^^from discrimmatory assessment has i^^^ Assessments made using 

prome-or^etfas in their present state of develdpmeritare hot 

. validated by anything outside the educational aricLperhapsi social values of the 
teachers in a particular sc necessary requirement that public discrimi- 

natory assessment be comparable across schools therefore: led the" researchers 
in the end to regard profiles and self-assessment, regretfully, as of only peripheral 
interest. ^ f 

l As it was considered that adequate information had be^ri^obtaihed about 
school examinations by the 1975 study^ 'Methods of Assessment ; and their 
aqciiracyz' tSj)encer, 1975), Course WorkaniContinuous Assessment remained. 
The research^team had been predi^osed£ol favour the former by the successful 
e>g>e_ri€nce5)f TWYLREB, in CSE assessment. In preliminary discussions some 
teachers were, however, asked for their reaction zto the idea of a 'continuous' 
scheme, ^arks to i)e recorded at monthly or two-monthly intervals The sue- 
gestioii met with no enthusiasm: it was felt that there would be too mudi 
administration arid^ top m^ with marks instead ofieaching and learn- 

ing. There was, on the other Jiand, cautious acceptance of or vvelcome for a 
'P^^o scheme m which O-g^^^ would be of ±e quality: of work 

in tpto, while each separate piece of work might be marked bn^bmplitipn for 
whatever purpose the teacher thought fit,_e.^. for immediate diagnosis of weak- 
ness, or for encouragement. Thus was the type of internal assessment settled. 

III. The Plaiiiied Folio Assessment Scheme 
(a) Agreed Teaching Aiins . ■ . 

The schools were invited to prepare^ the 0-grade English examination 

Jjy^orking towards the achievement of aims which the teachers recHomised: as 
implicit in the requirements of that examination. ^Occasional use of O-grade 
ppt papers or similar ready-made tests was admitted as necessary examiriatidh 
practice, but the teachers were asked not to Use them as part of their teaching * 
for the aimj_Mreed_jn:prd^^ meetings. The list of aims agreed by the 

teachers at preliminary meetings was as follows: 
WRITING 

Pr ose 

_ : Pupijs should be able to: 
FACTtJAt ~ brg^ise ractual material clearly and lo^cally, 

e.g. in reports, insimcybns, le^^ 
PERSUASIVE - present opinions, arguments^evidcnce. 

L : _ - write persuasively, to win support. 

MRSONAt - write about persdhal experience feelings^ describe DCKOnai 

tana _ interest, explore imaginative resources in response to various stimuli 

DESCRIPTIVE) describe scenes. 

PDETie • - vwite stori^. 

Note TOs categorisation is derived from J. Britton's * What's the Use' in Lansuake and 
Education, published for the Open University by Routledge and Kemn Paul Brittbh 

reiersto: _ . . ;: :: 

S^X |J^^?'<w>al' w^^^ to dir^t, question, Mt thin^ dojie, ^rticipate in society. 
(2) ^Expressive ^writing: to exchange or reveal: feelings and opinions, convey atti- 
tudes, reveal personality. ' ' .. 

« See Reports and Reporting, published by the Bosworth College, Leicestershire. 
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O) Tpetic' writing: construct iLnguistic_artefact&- organised language - as a 

means of Jrying to explore and the CPrnpIeAilie& oCxeality. 

The categories obviously merge into one another when people are actually writing. 
Poerry, :Drafm : . . . _ _ _ _ 

The writing of poetry and dramatic dialogue might also be rc^rded as aims of S4 O-grade work. 
READING 

Undentarciingandapprecimion 

U) Pupil^ should read as widely as possible and should devote sp«:ial attention to the 
. study oF a number of selected texts (prose, poetry and drama). This study should 
cjiable them to follow 4he narrative line, acquire insight into qharacters, respond to 
tone arid' to the attitudes and ideas presented by the writer, and recognise some of 
the elements in the text which contribute to the impression it makes (e.g.^humour, 
suspense, structiire, setting, images, dialogue, realism, etc.). 

The emphasis should be oh the pupil's own j^rceptibh of Uie meahihg arid tone of 
the text: there is ho advantage ih the ahalysis or labeHiiig briiterary devices vvithout 
some emdtidhal arid ihtell^tUal appr^iatiqh of ^leir irnpact^ and imp^ 
(2j Pupils should beLaMe to im^ of a_passag&oj*^propriate_difficulty 

which they have_ not : pre vipusly st udied^ and also the ideeper _or Jess^ jmmediate 
aspects of t he_meaning of such a passage. They sho uJd have, enough knowledge, of 
language to be able to indicate how the^ w^ been conveycdL They 

should be able to understand and summarise the whole or a substantial part of a 
passage of straightforward prose. 

(b) Metiiod of Assessment 

The pupils' work throughout the session was retained in a folio aiid assessed 
by the teachers in ways which varied slightly in differerit regions. Fife and 
Grampian teachers worked as departmental teams, assessing all their O-grade 
pupils as <>ne_ group ji in jStrathcIydei each teacher was responsible for his own 
class. The Strathclyde schools agreed to make three assessments - in late 
^^}^Tyj^M'tvd2iV^-m^ in Ajml - whileihe pthe^^^ were to make only ah end 
of course one im April: The. research team hoped to obtain by this variation 
s pme evjidehc^ abc>u t the relat iy e yalidit£ of single assessment asipj^ to 
cumulative or the average of several^ but the intention was frustrated by the 
inability of three of the five Strathclyde schools actuaHy^to carry out three 
assessments. 

In all cases, three marks were required; for Writing, out of 30, for Literature, 
out of 20, for Inteipretatjph,^ out.bf 50, these weightings being the familiar ones 
of the three elements in the O-grade examination: 



(c) ^Rules' and Minimum Contents of Folio 

Tihf Lidesij-a bili^ pS_3 statemen t of mm imum requirements^ and rules fo r des^ 
cribing the conditions under which work was: done had been established by the 
experience d^^ the GCE Joint MatricUla^^^ Board with its Alternative in^^ 
scheme. The 'Rules for Folio Assessment' which the SGRE project teachers 
agreed to are set out in the following statement, which was circulated to the 
schools: 

RULES FOR FOLIO ASSESSMENT 

The work carried out by the IJupils ih the course of the year might arise from various approaches 
to the agreed aims - theme work, uriit^ study work, or whatever the teacher might decide. It is 
proposed that all thepupirs work towards the agreed aims be kept in the folio, and that the 
following be the minimum requirements: 

NVRITING • _________ 

(1) Each pupil should complete hme assignments of 350-450 words (or the equivalent 
iivleriithicf pi^es), . _ _ _ ^ 1 ' 
Stx of th^e should be chosen By the pupil and the teacher for assessment, the 
selection to contain evidence of the papirs ability in the categories of Writing listed 
in ^Agreed Aims for S4-Enj^ish': 

(2) Each composition chosen for assessment should have the robric of the assignment 
attached to it, and, if possible, any stimulus material, or a description of the stimulus 
material: 
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Each camposition sfaotdd be Jabelled ~ 

(a) *lslilraft' or *improyed draft*; _ 

(b) * Lfiiaided' or 'Aided': the Jype of aid should be Briefly specified, e,g, Tbllbw- 
ing class discussion'; or, *After other pupils' criticism'; or, 'With some 
guidance fronx teacher'^elc, 

L (c) *Ciasswork' or *HQmework.' 

(3) There is hb heed tb avbid chbbsihg for assessment work improved by discussipn or 
criticism, so Jo-ig as all the work chbseh can be said to represent the pupiPs real 
abUity in^ English. However, the selcctibh shbuld include at least one unaided firet 
draft of an assighitieht done in class. 

READING 

(1) The^papils should k^pa record of their reading as biitlined bh the specimen *Recbrd 
of j^eading' (see below) and this should be kept in the fblib. 

(2) l;ie folio should contain at least j/jc pieces of evidence of the pupil's ability to under- 
stand and respond to seye^ 

Clhterpretaiiori' questions on sections of the texts studicd_come into this categoiy.) 
^bwrof these pieces of work shbiUd be chosen for 

The folio shbuld in addition cbritaih some general comment Indicating the^ifpU^s 
W^^^Ji^^ prose, poetry and drama he has studied. 

(3) The folio should contain a minimum of^tx *anseen' interpretatibhs of the traditibhal 
type. „ _L : : L 

/^pwr interpretations shPiUd be a^ including two which required the pUpil to 
summarise all or part of the passage. 

(4) In each case, work tor assessment shbuld be labelled 'Aided' or 'Unaided'^ 'Home- 
work' or *Classwork', as tor 'Writing' above, and at least brie of the 'unseen' inter- 
pretations chosen for assessment shbuld be work dbhe uriaided iri class. 

SPECIMEN RECORD OF READING (O-grade English course, August, 1976 - April 1977) 
Name: School: 

Class: 

(1) l^xrs smdted as parr of school work, (Titles arid authors) 

Poetry. \ 
._ Dr ama 

(2) Texts redd In additio/i to the above ^ (Titles arid authors) 

(Ybu may iriclude here any stbry, bobk or poerii you have read^ any pjay you have 
read or seen performed. If ybu are dbubtruf abbut the suitability of a text you want 
to include ask fbr ybiir teacher's advice.) 

ORAL 

(N.B. This part o^the scheme is optiona^^^ in the comparison between 

the school assessments and the two cxternaljexaminations.) 
: Teachers will be asked in January and in April 1977 to give ah Imisressiori^adirig' bf pupils' 
oral ability,^ based on general criteria to be Supplied by the Research tearii. If possible, teachei^ 
oiher than English teachers may be asked to provide ah oral assc^smerit gmde tbb, sb that the 
reliability of this grading might be increased: 

Support for Teachers _ 

Two. means of helping teachers with aisisesisment problems were adopted: Trial 
Marking and provision of a Guidance bbbklet. 

zTrJaL Marking was mod^^^ by The West Yorkshire 

and Lindsey Regional Examining Board (TWYLREB) for the CSE^ Four meet- 
^fljs pf school r^re^^^ piMned for June and Septe^^ 19^76^ and 

Januiiry and March 1977, for the purpose of establishing common standards 
on scripts selected by th« researchers and previously^ se^t to each school for 
the department, or at least three members of it, to mark. The representative 
thus was able to bring to the meeting his schoors assessment, arid not riierelj^his 
own. Although only three Trial Marking meetings were actually held, this part 
of the work of the project was very successful: it is reported in detail iri Ap- 
peridix 5. 

: Th^e^Oiiidajiee bp^ of advice about wa^^^ bf improving 

the validity and reliability of assessment in English, suggested criteria for mark- 
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ing certain writing assignments, and exemplars : of tests br rcsporisc to short 
stories in which the questions had been derived from suggested 'purposes of 
rcadihg', following a model created by the Reading Peydgpincnt Course t^ 
of the Open University^ The booklet wa;s late in preparation and too bdky, so 
that any effect on teachers\assessnient during th was probably miriimal. 

[t is hoped that its recommendations may be helpful to the teachers in the 
future*. 

(eJi Mpderatibn : : ^ 

The feasibility and effectiveness pf two methods of moderation were tested. 



I. Visiting Moderator : : : : 

Moderators were appointed to visit the scIiodIs on two or three occasions 
to monitor the internal assessment scheme^. Their functjon^^^^ to discuss 
with teachers difficulties ^irising, iji particular those concerning the com- 
parability: of levels of difRciilty of avotJc set in jdiffereht schools (a ^ 
of trouble to the JMB internal assessment scheme), (Li) to moderate the 
standard of the schbSl assessments by marking the folios of a sample of 
20jpupils: 

The statistical tests of comparability between school and modenitor 
were those of 'Range estimates' developed for CSE moderation, described 
ill ^Schools Council Examination Bulletin No; 5: The Certificate of Secon- 
dary Education: School-based examination's\ and used 
The application of these tests in the SCRE project is described in the 
discus.sion of moderation in Appendix 6.: : 

Two types of moderator were employed. It was assumed that in a real 
internal assessment sys vvould be experienced aiid sUcc^ 

fui d-grade markers (wife le examiners would probably be senior moder- 
ators). Accordingly, it vvui.s decided to appoint four experienced markers 
to Fife and Strathclyde,]each to deal with two or three .schools.. In the 
Grampian region a teacKer from each school, who was involved in the 
scheme and had taken full part in the Trial Marking exercises, carried puj 
moderation of marking in another school in the group: The teachers in 
this groui? did riot wish tp: take on the t^ask j3fj\iKiging ^w^^^^ 
by colleagues in other schools was of the appropriate level of difficulty, 
so one of the researchers Uridertbbk that part of the moderators' duties. 



2. Statistical Stjiridardising - i '- - 

The marks given by the teachers were scaled against those obtained by 
the same grbUps of pupils in both the O-grade and the Criterion Test. 

(f) Modifications to the planned Scheme 

The experimerital scheme: as lagreed with the M^i^^^ eairly 
Summer, 1976, did not proceed without modificationr One Fife and one Strath- 
clyde school withdrew^ in Autumri, 1 97^, frbrn the commitment to pf bvidc a 
folio assessment: Another Strathclydeschool maintained its intention to provide 
cpiirse^work marks untH March, 1977, but was not in fact able to do so; and a 
third member of the Strathclyde group fulfilled the coriiriiitmcnt only p^artia^^^^ 
sending marks for two classes but of six. (The remaining two Strathclyde 
schools co-operated fully and did each prpyide threeiasscssmem^ 
of the year, as requested; no attempt was made, however, to investigate the 
relative coriiparability: with the Cn^^^ Uiree assessments se^ 

rately and combined, as it was felt that too many of the pupils for whose results 
this study had been planned had been Ibst through their schools' inability to 
provide marks:) y 
' The Guidance booklet is under revision for possible wider circulation: 



[ri the schools which fuifilled all the demands of the project, fears arose carl^ 
in the session that the agreed minimum^ re<tui>emeht for the foHos wottid prove 
more thai! cou^ a significant proportion of pupils: Several teachers 

who were trying to conform with the suggestion thatpast papers might not be 
used as tea^^^ anxious about the difficulty of finding or 

inventing alternative interpretation materiaKjn the very limited free time: they 
had; they were also wprr^^^^ about the need for pupils to have examination 
practice. In the event, almost all the pupils involved probably did pFodiice the 
minimum amdunt of workup some doubt, because in 

seiTie schools pupils were allowed to remove material from' their folder^z to 
revise for theO-grade examination before thc: moderator scrutinised the school s 
assessment.) The teachers' concern in the early stages? was probably due to 
msectirity in dealing wjth an Uhfagtiliar system and jo the {eeling that they 
simply did npthave enough t to teach for O-grade and try to engage in 
some curricuJura development as well. In October. 1976^ ther-esear<:h team sent 
to the schools the J>opkIetioFg^ on internal assessment, the contents of 

which may have caused more confusion than enlighjerimerit: it cmnained siig- 
gesttons for diagnos^^^^ for types of English work not normally 

undertaken in preparation for ©-grade,i^s well as genera! advice rejarding: the 
vahdity and reliability of assessments. Though this booklet was meant to be a 
seJzpTideas which teachere might /try out or not,^as they wjshed. it was mis- 
understood by some to be a very late statement of more 'rules' which they felt 
they had -not ha<lj time to consider arid plan for. As a result of reports from the 
schools referring to this booklet arid to the fears described abovci the researchers 
drculated in lat^^^ 1976, an explanatory leaflet entitled 'Some Clarifi- 

cations', to reassure the teachers. 

_ Qarifica^^^^ of the experiment as (i) 

_ i^sessment of a folder of course work arising from the schbdl's teaching jovvards 
O-grade aims, (ii) the cpriiparisbni_6^ that assessment with 

t.vpse awarded in our Criterion Test and in the b-grade cxamiriatibri. Reassur- 
ance was given that the agreed miri^murii requirements, the request to avoid 
past papers, and the suggestions in the Guidance booklet were all subject to 
the teachers' judgment about what was best for their pupils and yvliat it was 
possible for the p^^ do. Adjustment of tlic 'rules' vvas.pcrmitted to this 
extent and it was pointed out that one of the purposes of the project was pre- 
cisely to report bri the amount of Work jt was reasonable to ask of pupils and 
teachers m the course of a year: The document also repeated the assuranec 
that moderation wbuld ribt in any sense be an inspection of tlic quality ol^ 
teachers* work. 

^ There were three further altcratibiis to the originally projected scheme, besides 
those allowed: !^ Since it became obvious that everybody 

was finding difficulty In providing the basic necessary data, two suggestions 
writtcri iritb the plan were quietly, though regrettabiy; dropped : these were the 
invitation to assess the pupils' oral work and the request for a record of the 
Jjupils' reading during the year. The third oini.ssioh was of the Trial Marking' 
exercise scheduled for January, 1977. 

Detailed accounts of practical problems which occurred and of teachers" re- 
actions to the project can be found in Chapters IX and X. 



(g) Summary of E?^^^^ 

Table 2.J sets out thc_schcmc in full, indicating the priheipal modincations 
described m (f) above. The time-seal^ for the preparation and implementation 
of the programme are shown in Tables 2.2 and 2.3. 



22 



TABtE 2:i : FOtlO ASSESSMENT SCHEMES (SUMMARY) 
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TABLE 2.2: TIME-SCALE FOR PREPARATION OF FOLIO ASSESSMENT 



^ Fehrudry 1976 


March/ April 1976. 


June 1976 


AiigUsJ/SepTember 1976 


1 St cofifact 
with school 


(1) Preliminary Visit to school. 

(2) Teachers consider written 
proposals. 


1st Trial 
Marking. 


Implementation 




(3) Second: yisitio receive 
objections, etc. 




\ 
\ 




(4) Meeting of school 

representatives to finalise 
plans. 





\ 



TABLE 2.3 r TIME-SCALE FOR IMPLEMENTATION OF FOLIO ASSESSMENT 



September 
1976 


I _ _ 

Oaober 1 November 
1976 i 1976 

1 ft 
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February 
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\ , 

] 

' 1 


(U 
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Modera- 
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1 school 
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Trial 

Marking 
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(2J 4ih 
Trial ! 
Marking j 



I I if"rt"" i , Modera- 
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CHAPTER III 



MEANS OF EVALUATION OFTHE INTERNAL ASSESSMENT SCHEME 
I. Overview 

The Folio Assessment scheme^ was evaiuated statistically as regards ranking 
and as regards judgment: of D-^rade stand^ards: at jeac Mndiibmmda^l by 
comparing the macks awarded by the Folio Assessment and im the -Q-grade 
exarriination w same pUgils in the SCRE ^C^ite^^^ 

Test', an extensive, thoroughly prepared and reliably marked ind^endent as- 
sessment. This statistical evaluation is reported in Chapter TV. 

Chapter V deals with the methods of moderation tried dUt in the pfbject, 
corisideririg their efTectiveness Tor imposing O-grade standards on internal as- 
sessment and also their effects on awards to Individual pupils. 

Injhe course of comparing the three sets of marks it was realised^that certain 
related, maUers were: pF considerable: irhpprtanc^ in trymg tp decide whether 
Foiioi Assessment is as good as or better than^the external examination. Chap^ 
ter VIxljscjjsses some of these raatters -: in p^^^^ Siafker unreliabilit£ 
the problems caused for comparability of assessments by allowing variation in 
tfc^MM earned o^^^^ 

and Literature, on the one hand, and Interpretation, on the other, are pointed out. 

In addition to purely statistical concerns the researchers were interested in 
the practical feasibility of an internal assessment scheme and in the educational 
advantages and/or disadvantages. The difficulties met in the implemeritatiori of 
the scheme are described in Chapter IX, and suggestions are made as to means 
pr red uc ing t h em . Pract ica 1 pro b 1 em s also figure :p rpm i nen t ly : in Chapte r: :X ^ 
which is devoted to the views pf the- teachers who took part in the experiment, 
but_ another: jr^^ this.chagter is to present the teachers* 

opinions about the educational value of the scheme. 

It was not possible in the project to make any kind of direct assessment of 
the am o uri t of progress pUp i Is made as compared with what would h av^ been 
the case in a normal year. It also proved impossible, despite the good intentions 
of the researchers, to survey the views of the puiJils themselves about the project: 
very 'few were available to be questioned in the third term, after the O-grade 
examination. 



11. Statistical Criteria of Evaluation 

Folio Assessment could be statistically evaluated as vegdvds ranking and as 
regards judgment of O-grade standards at each band boundary. 

(a) ^mkmgiikefiincitm o^ 

It mayj be: Mfei^L to defi ne _t wp well -kn own \>u t st i 1 1 important ch aractenstjcs 
of good assessment. (I) It should be valid - i.e:. It should actually test what it 
Glaims to test: the extent and qualiy^ of pup^^^ ^I^E^^^^rX?) \^ 

should be reliable - i.e., it should be as free as possible from chance factors 
which might influence the marks awarded. 

In the project's study of internal examltiatlons in 1975\, the school assessment 
was compared only with the O-grade results. In order to compare the relative 
qualities of Folio Assessment and the O-grade examination, an indgjendent 
criteripji: was required, whic to be a more: thorough and reliable 
ment than a normal examination, because of the known Imperfections of tests 

' Reported to SCEEB in Spencer (1975;. 
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like the O-grade English cxaminatidn. For practie^I J^asons, such public ex- 
amination's sufTer a loss orivali because they can sample only a small pro- 
portion Qf a candidate's attainments. They may also, d_espite c_arerul preparation 
and moderation, occasionally j>resent candidates with, for instance, questions 
wmeh_4re invahd because of ambiguity of wordmg which was riot detected until 
large ntimbers of pupils actUpy^ Or, from time to time, 

a passaje fcH-jnteipretat be chosen which requires speciaHsed know- 

ledge^ from anyone required to Uriderstarid it ruUy. Pre-testing of the papers 
would: bring such faults to light, but is impractical for annual 0-erade 
examinations. 

The principal means of reducing the UrireIiabiIity_whLch remains in an Engiish 
examination even after markers' meetings - multiple-marking to reduce irtter- 
. rripxker inconsistency - is also normally impractical^ so that a degree of un- 
rehabihty is inevitably presqrit in an O-grade examination: 

^ The same - imperfectibri^ iriflu^^^ assessment such as the Folio 

bc|_e_me employed in the SCRE project, though their relative si^ificanc^ is 
dinerent. tn folio ^assessment there js^^^^^ than in 

an external examination^ because more work can be assessed, but there are 
hkely to be more difficulties with reliability and with comparability of work 
and standards of assessment in different schools: 

The function of th^ Criterion Test was to provide> Tor _thc purposes c the 
c^eriment, a measure of pupH^^^^ English which would combine 

tf^g ejctensiveness of Folio assessment with the comparabiliti^ pf_ an external 
examination and which cbuld_be_i^^^^^ prepared and marked than 

a teacher s assessment or even an O-grade examination. Accordingly the re- 
quirement of the remit of the proiect was that the researchers should create a 
comprehensive test of all the 'objectives' of Enghsh teaching in S4 preparatory 
to O-grade; optimise its validity and reliability, and use the marks given in it 
as the ^criterion' ajainst which both the internal assessments and the O-grade 
results might be compared. The test was to be a 'criterion' in this sense only: 
It was not a 'criteribri referenced test'. 

TlKdevelqpmerit bf the Cn some theoretical and practical 

problems which, since they have their own interest, have been keptfor a separate 
section, of the report: the detailed account of the making of the test can be 
lound jn Appendix 3. Pages 122-124 contain a statement of the ranje of skills 
and kinds of English work which the test covered and also of those which were 
cxcliided for various reasons. The test papers themselves and the marking 
schemes can be found in Appendix 4. It is eribugh to note here that there were 
h ve papers, coveTinfe: free composition with picture or .'imaginative' verbal 
stimuli; two writing exercises with specific purposes and foir stated audiences; 
two- passages for /tradiUoM three others with multiple- 

choice questions; a test of close reading of a complete shbrt stbr^, with guidance 
as lo what to look for arid (within reason) unlimited time to answer; and three 
questions requiring 'general' responses to, respectively, a pberii, a play and a 
prose work chosen by the pupils themselves. 

'Validity' in respect of this test riiearit brily ^ace-validity', but as high as 
possible. T^ a test on which two of the researchers, bbth 

experienced teachers, had expended much thbujht^which had been pre-tested 
where appro through critical evaluations of its 

suitai)ility for testing the English of 16-year-blds by O-grade examiners and a 
number of other teachers. Throiighout the consideration of validity and of 
Enghsh 'aims', 'objectives', 'criteria', 'skills' or 'purpbses', the project rejied for 
justification of the rightriess of descriptions, definitions, test questions, and 
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passages, bri the experience as teachers and examiners of the researchers and of 
ii fairly large-group of others lnvolved in teaching and examining whose views 
were sought frprri time lb time tThc names of those cbntritjuting to thej project 
in this way arc listed in Appendix 2,) The validity of the Criterion Test as an 
iristruriicnt for assc^ssing as tru^^ the achicyemchts in English of S4. 

pupils was therefore underwritten by several knowledgeable Engh'sh teachers 
'^'^^MpPTS w their approval to, or suggested modifications in, its 

various elements. The test could be said at least to match the O-grade in respect 
of the care and experience employed ill its making and to improve Upon the 
SCEEB examination in its extensiveness. 

: The influence bn the results of the Critenbn Test bf chance factors was re- 
duced as compared with O-grade or FoHo assessment, because, besides the 
nbrniaj inarkcFS-m special measures were ^ in marking it: (I j all 

writing and literature tasks (i.c.:^ Papers I, IIF, and V) were doublc^markcd by 
experienced O-grade markers ; rurther, ^hc ib^jgation on tire piipi I s_ to wrUe 
three compositions, two of which were for defined purpx)ses and audiences, and 
to rcspo n d to spec i ft c q u es t jo h s on a set tex t , as well as to ' open ' I i tera tii re 
questions, was Hkcly to increase the reliability of marking, as against an ab- 
solutely free choice of cbmpdsitidn and of literature texts and questions; (2) 
marker-inconsistency was removed altogether from Paper IV, the multiple- 
choice test, which had been subject to vetting by a team of question setters and 
had been satisfactorily pre-tcsted; (3) the procedures adbpted to standardise 
markers' assessments in Papers I, 11; IIF and V - adjustment of both the mean 
m>vri<ji warded by each marker and the spre^^^ more 
unreliability than the SCEEB- procedure, which adjusts the mean only: (As will 
t^c seeri in the discussibri of the corrclat[bn cGejflcients^ >yas howeyer 

one school, 13, in which the Criterion Test was not set in the standard way. 
With thtf result that its reliability for that .school is more question able.)' 

Though it would riot always be safe to assume th^t Folio assessment and an 
external. examination are measuring the same things, the school assessment, the 
O-grude aiid the Crllcrioii Test in this case did have common teach uig aims aild 
criteria of: achievement. The similarity of teaching afm.s is evident if the Folio 
assessments aims (stated on page 18) are compared with the tasks set in: the 
^ ri ten on Tes t {^ce page 1 22) , bo t K ha vj ng been d cri vcd u] timately frorn afi ajy&j .s 
of O-grade questions and markerii' Instructions. Marking criteria and processes 
lor the Cri terion Test were mpdcncd oh th c O-grade ones an d O-grade riiarker.s 
were used; the Trial Markings, in which the teachers involved practised assess- 
ill eri I, al.sb applied O-grade criteria to the sample scfipt.s. On the basis bf the 
con sicierat ions of validity and rejiabijity described above, the rank order pro- 
duced by the Criterion Test could be taken to be the best of thc three available. 
A pcvsitivc cbrrclatlbri between either of the other twa sets of marks and the 
Criterion Test marks rany be interpreted as indicating the degree to which that 
test ari d the Criter i on Tes i are rcl i a biy mcas li ri ng j he same t h] n gs . If t he cp r- 
relation: between Folio and Criterion Test Ls significantly better than that be- 
tween O-grade and Cri teri bh Test, thc^ Fbl ib assess m eri t can be sa id tb have 
reliably assessed more of whatever is measured by the Criterion Test t' m the . 
O-grade did. The, relative merits bf Fblib arid O-grade as regards. rariJcihg the 
pupils iValidly and reliably were thus judged by comparing the correlations they 
both obtained aguin.st the Criterion Test. 

(hj Juc^meni qf 

' The criterion of standards, or of the appropriate levels in the rank order above 

' No reliability cocnicicril was calculatcU lor Uic Crilci^ibh Test because it was Uibuglit iKal, 
since each paper sought to test at least some ditfei cht skills, a split-Half method of calculating 
one, or any variation bf it, was inappropriate. 
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which to award bands A', Bre, D, and E, had to be the judgment of the O^radc 
examiners in assigning fanges'arid baiKls to pupils in the examination: the 
researchers epiild not guarantee to set standards for the Criterion Test equivalent 
to those of theiO-grade. (Hi the event, the Criteripn Test as set and marked was 
probabiyj iittjejniore severe than the 1977 0-grade examination.) Accordingly 
performance in the'O-grade was the basis of conyjarisbriiwhen standards were 
considered, and the proeedureiadopted was to compare Folio and O-grade pass 
rates at each band before and after the scalinjg of Fblib marks against those 
scored by the same pupils in the external examination. 

III. Note qa Terminology : Marks, Standardising, Sealing, Ranges 
and Bands 

iWarA-y rriay be *raw\ *standa_rdised\or 'scaled'. 

:i _8^aw marks are those actaally xiwarded by ihe markers to scripts. 'Standardi- 
saUon^ usually means re-e>^^^ in some standard way :^ e.g., all 

p-grade raw marks in all subjects are standardised to a pass mark of 50% and 
a st^dard deviation of 20. The term 'scaHni' normally refers to the process by 
which a Mp^ raw marks is adjusted so that its mean and standard deviation 
are the same as. that of another set of marks for the sameipupils - e.g., raw 
awarded in tha assessment and in the Criterion Test could 

be scaled against the pupils' marks In the O-grade. 

Some confusion is possible, because SCEEB officials and examirierseommonly 
use the word ;scalin^\iri re^^ described in the previoDs 

paragraph, while they use 'standardisation' to refer to a different process, the 
one by which each marker's consistency arid accuracy is checked by the Principal 
Exariiirier and his team. 

in Ms rej>ort, the following definitions obtain, unless the text specifically 
indicates otherwise: 

\.Ra\v A/;arisr jnarks actually awarded to scripts by markers^ i 
^^^^or^fsoUoh^^^ of raw marks so that the pass mark decided 

on by the Principal Examiner-becomes 50 % and the standard deviatibri of the 
mdrJ^ becomes 20. (This is often-called -scaling' by SCEEB.) 
ScaThm; the adjustment of a set of marks so that its mean and standard 
deviation are the same as those of another set of marks for the same pupils. 
MarJier^M^ the process by which the Principal Examiner and his 

colleagueS\eheck the consistency and accuracy of individuaf markers' work 
arid make neeesMry:^^^^^ 

/?a/fgesartaf^i/y- standardised categorisations of riiarks employed by SCEEB 
in reporting exHmm^^ between SCEEB standard- 

ised marks and ranges and bands is shown in Table 3.1. 

\. 
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TABLE 3;l: MARKS, RANGES AND BANDS 



SmndardUed mark 


flange 




90-100 


I 




85-89 


2 




80-84 


3 




75-79 


'"4 




70-74 


5 


A 


65-69 


6 




60-64 


7 


B 


55-59 


8 




50-54 


_ _^ 


_ C 


' 45^9 


10 




40^ 


M 


D 


35-39 


' 12 




30-34 


13 


E 


0-29 


14 


No 



award 



Note: In the SCRE project, for convenience, 'No award' has been called band 'F': 
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FOLIO AND O^GRADE: RANKrNr AND PASS RATES 



I. Ranking - Comparison of both Folio and O-grade with Criterion 
Test 

(a) Overali Tmpression - 

Pearson:^ Prpducf-mo correlation coeificients were calculated for Fblib/ 
eriterion Test, O-grade/Critendn Test, arid Fblib/^ 

The three coefficients could be obtained for II schools; of the remaining 
three schools, twb^priovided no folio assessments, so that only the O-gfade/ 
Criterion Test coefficient was found for them; and Grie_did not set the Criterion 
Test aiid had to bejeft out of thecalc^ Some figures are available, there- 

fore, for 13 schools, all figures for 11. 

In order to ^ive a general impression of the Validity, bf each of the three 
assessment methods by cbmgarisori with the other two. Table 4.1 shows the 
coefficients R)r all the pupils in the i 3 schools for whom daf a had bee^i bbtained. 
ft should be noted (1) that the three coefficierits are not strictly comparable 
becaiise they were calculated for three slightly, different populatidris, (2) that 
the quality^of assessments in individual schbbls and classes cannot be judged 
from these cprrelatipns. In very general terms however, FoUa Assessmen^^^^ O- 
grade and the Critermn Test can te said to corretate Mth one another satisfactorUy. 



TABtE 4:1 : PEARSON CORRELATION COEFFICIENtS 



r Folioi Criterion Tesi 


O-gradelVrTterTdh Test 


FolialO'grade 


•77 (1567 pupils) 


•82 (1834 pupils) 


•71 (1529 pupiis) . 



Jud^ents about the relative merits of Folio Assessment and O-grade ebuld be 
made only after cdrisidefatibri of the cbrrelatibns by school and by class which 
are given in Tables 4.2 and 4.3. 

Interpretation of correlation coefficients is not stEaightfOfward: the value of 
the coefficient depends, not drily dri thenjuality: of the assessments, but also 
on the srze of thejiample and the distribution of marks within it. Thus a schddl 
presentmg for O-grade and Folio Assessmerit drily thbsejjupiis likely to obtain 
band C or hetterjs_ likely a lower correlation coefficient th^ dne 

without restrictions on the type of pupils sitting the exariiination^ the quality 
of assessment being equal cannot, therefore, read 2/oh77 Table 4.2 

reaching conclusions about the relative quality of schddls^ Folio Assessments 
simply by comparirig the cdeffi^ different schools. Caution is 

needed. The Notes columns of the Tables provide informatidri df which account 
should be taken in evaluatirig sdriie df the correlations. It is however, legitimate 
tbimakeiCbmpaFisons across tdich row in the Tables, since the pupils irivblved 
are the same (or virtuaUy_so). The size df the Fblio/Griterion Test coefficient 
relative to that fg^^ indicates the degree by which Fdlid 

Assessment was better or worse than the O-gfade at rarikirig the pupils on those 
skills measured by the Criterion Test. 
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TABLE 4.2: PEARSON CORRELATION COEFFICIENTS: SCHOOLS 
ASSESSING ON WHOLE-SCHOOL BASIS 



School 


r Foliaj 
CrUetLon 
Test' 


j r. O-gradef 
1 CrUezion 
Test 


r FdUo^ 
O-grade 


• 

NoJi^s 


1 1 


•74 

(144 pupils) 


•69_ _ 
1 (143 pupils) 


•68 _ _ 
(143 pupils) 


(IX Fjalio assessment made m 
ranges J^14, nol in % jiiarks- 
so tanking^was less refined 
than elsewhere. 

(2) Truncated : poorer pupils 
not entered. 


12 


hb folio 


•73 
(99)- 


no folio 


Part of Cri terion Test Jakca.- 
|n class: some pupUs therefore 
failed to complete it. 


13 


CI 63) 


•85 
(165) 


•61 (163) 
•71 (262) 


Part of Criterion Test taken 
. in class: 165 pupils only 
completed the full test^ 
though 262 sat O-grade and 
received Folio assessment. 


14 


(227) 


•51 
(218) 


-74 
(196) 


Truncated at top: best pupils 
by-pass O-grade. 


15 


•84 
(155) 


^84 
(154) 


(154) 




21 


•86 
(94) 


•82 
(93) 


^r-^ -83- 
(93) 


Truncated at bottom ^ poorer 
O-grade pupils excltid^ from 
project by school: 


22 


•84 
(204) j 


•85 
(209) 


•79 
(204) 




23 


•84 ; 
(81) j 


•87 i ^79 
(89) ! (81) 




24 


89 f 
(72) 1 


•88 i 84 
(70) 1 (70) 


f 



Nbter School 12 dropped out of the Folio scheme early in the session. 



The picture presented by seven of the remaining eight schools carrying out 
whple-schdol ass^^^ a satisfactory oh^^^^^^ the J^plio/Criterion Test 

coefficients are better than -86. Tiie lower value for school 11 (-74) may be 
partly explained by two factors: (a) the Folio assessment was report^ to SCRE 
in discrete ranges 1-14 and not In continuous marks I-lSS; (b) the distribution 
of ability in the sample was restricted by a policy of riot ent^mngj^porer 
pupils for the exammatidn and by the relative lack of exception^ly lii^ achievers. 
(Of 38 pupils in school II obtaining O-grade band A, II had ranges I, 2 or^, 
as a^ihst 14 out of 30 in school 12 and 16 out of 31 in school I5,:two nearby 
schools.) 

: iT!ie - t' test for si^^^ two correlations which 

are also correlated with each other was apphed; 

In seven ojf the eight coses of* Wh&te^school ossesshiehi there is no signijficcint 
difference between the FoJiojCriterioh. Test and the O-grade/ Criterion Test cor- 



retgiTmsy JIiese scTwoTs can be said to have ranked their pupits sartsfacrorily by 
comparison with an m (o Tigve done so as well as the D- 

grade examination did by comparison with the same criterion: - 

case of school I J^ the superiority Of the Q-grade/Griterion Test cor- 
relation over the Folio/eritcrion Test correlation is highly sighificaht. Possible 
reasons for the distinctly j>^^^ Test correlation in this school 

are suggested below (see page 35): 

ie^ L Fpiip Assessment on a Ciass iasis - i 

Tables 4:3 and-4.4 show the correlaUqhs for the Strathclyde schools, which 
had presented Folio Assessments carried oat by teachers for their own classes. 



TABLE 4.3: PEAMQN CORRELATION COEFFICIENTS: SCHOOLS 
ASSESSING BY CLASS - SCHdOL CORRELATIONS 



School 


r Fotiol 
Crtierton 
Test 


r O'grade 1 
CrVterJon 
Test 


r Fqlioi 
O-'grade 


' Notes 


32 


•83 

(268 pupils) 


; (270 pupils) 


•71 

(268 pupils) 




33 


•55 

[ (45) . 


•84 (126) 
-71 (45) 


•39 

1351 


1 Folio as5<^rnehts for two 
1 Glasses only. 


34 


no folio 


•86 

(85) , 


hb T6\\6 


1 Many pupils did only part of 
! the Criterion Test: 
1 85=brily about two thirds 
1 of the presentation grdtip. 


35 


' -63 
(114) 


:84 i 
(M3)_ 


' •55 
■ (112) 


1 Sdm&class^ not Included in 
project, because of teachers* 
1 unwillingn^, but remainder 
1 cover a wide mnge of ability: 



^ School 34 was unable to provide folio marks, and school 33 could provide 
only_partiaI data^ (The differenceibetweeri -55 for FolioyCriterion Te_st -71 
for O-^rade/Criterjon Test for 45 papils in this school is not significant.) Of 
the^two Strathclyde schools providing full datai school 32 has hp si^jficarit 
difference, between: the correlations and school 35 has a Folio/Criterion Test 
correlation significantly lower than that for O-grade/Criterion Test. 



TABLE 4.4: PEARSON CORRELATION eOEFFICIENtS: SCHOOLS 
ASSESSING BY CLASS - CLASS CORRELATIONS 







r fbiiai 


j r O ^grade 










Criterion 


Criterion 






School 


1 Class 


Test 


Test 


O-grade 


Notes 


32 


\ al 


•75 (25) 


•59 (26) 


1 63 (25) 


Ranges obtained in O-grade 


First half of 










-1-9. 


alphabet,. 


! a2 


•8j (24) 


•66 (24) 


•58 (24) 


Ranges obLained in O^rade 


ro ughly se t 










= I-I3. — Foljo__assesscd_in_ 


by preyipus 


i 








: ranges not marks. 


assessmen ts 


1 a3 


•8j (27) 


•48 (27) 


•32 (27) 


Ranges obtained in O-grade 


(in English) 










=4-13. 




1 a4 


•35 (24) 


•72 (24) 


•27 (24) 


Rang^ obtained jn O-^de 
— 6-14. - Folio assessed in 
ranges. 




a5 


•48 (24) . 


•63 (25) 


•30 (24) 


Ranges obtained in O-grade 
^8-14. 




a6 


•78 (13) 


•75 (13) 


•78 (13) 


Ranges obtained in O-grade 
^10^14. 


Second haJt* 


bl 


•64 (26) 


•60 (26) 


•47 (25). 


Ranges obtained ir. O-grade 


of alphabet 








= 1-10,, . _ 


rdiighly set 


b2 


I -57 (26) 


•39 (26) 


•41 (24) 


Ranges obtained in O-grade 


by previous 










-=3-12. - - - 


assessnlents 








•t)J (27) 


Rang^ obtained in O-gradc 


(in English) 










=^3-13: 




b4 


•59(25) 


•46 (25) 


•39 (24) 


Ranges obtained in O-gfade 
==5-14: : 




bS 


•77 (20) 


•72 (20) 


•74 (24) 


Rang^ obtained In O-grade 

==^14: : 




b6 


•92 (fCi 


•81 (K\ 


•78 (13) 


XXcUl^VO- WUUUAAwU 11-1 S,y— 2^ mUC 

= JJL-14..- Pupils vodely: 

^ca.ttercd withm mnrk^ 

range 0-44. 


33 


1 


no folio 


•65 (27) 


no folio 


Ranges obtained in O-gradc 


Classes set 








= 1-9. 


by previous 


2 


•39 (5) 


•63 (25) 


•81 (5) ' 


Ranges obtained in O-grade 


English 










=j-IO. 


assessments 


3 


•55 (21) 


•50 (21) 


•03 (2j) 


Ranges obtained in O-grade 
=2^13. 




4 


•45 (i9) 


•6j (19) 


•33(19) 


Ranges obtained in O-gradc 
=i5^14. - Folio assessed in 
ranges; 




5 


rib rolib 


•73 (18) 


hd Tofib 


Rahge.s obtained in O-grade 










= 6^14. - Fblib assessed in 
- rahgcSi 




6 


ho folio 


•49 (16) 


no folio 


Ranges obtained in O-grade 










-7-14. 


35 


1 


•49 (27) 


•50 (27) 


•36 (27) 


Ranges obtained in X>-grade 


Classes set 










^I-5.-(+l range 7^ 


by previous 


4 


•75 (24) 


•47 (22) 


•56 (22) 


Ranges obtained in O-gfade 


English 










^1-14: 


assessments 


5 


•61 (20) 


•77 (20) 


•49 (20) 


Ranges obtained in O-grade 
=3-14. Ill 




7 


•71 (21) 


•69 (21) 


•65 (21) 


Ranges obtained in O-grade 

i: 




8 


•46 (12) 


•76 (12) 


•64(12) 


Ranges obtained in O-grade 


1 








- 5-13. _ 




9 


•82 (lb) 


' •83(11) 


•64(10) 


Ranges obtained in O-grade 


1 
1 








=5-14. 



1 Since three of the five schools committed to assessment \5y the class teacher 
'did not provide adequate data Jud^ of this method 

are not very firmly based. There are, however, some indications that class 
teacher assessments are not adequate as a basis for public certification. Dlffi- 
cukjes in moderating class assessments, either statistically or by marking sample 
scripts, are obvious. Some indications of 'class effects' even in^ the 'whole- 
scJiboIVassess^^ to the conclusioji that team assessment is 

desirable to counteract teachers' idio^ncrasies in setting tasks or marjcing. The 
common practice of ^^^^^ 'ability groups' can cause 

class assessments to be more inaccurj^te by comparison with a criterjon than 
they might have been: teachers may ran^ well within their own 

elass but be influenced in making awards by the supposed 'quality' of the set^ 
^^^^^^^ J^^^^^^^^^^^ award only Mn^i^A to the top class and nothing 
abj)ye band E or D to the bottom one. This may result in a very unsatisfactory 
rank order for the whole school, since the assessment^ orii which setting was 
based in the first place accurate enough to assign pupils 

with certainty to different ability groups. Though there may be faults in O-grade 
J'^nking too, the bands ach^^^ in the examination by pupils in a particular 
class are a fair indication of Ihc breadth of achievement in itv ft can be seen 
from the \Ndtes' column ^o^^^ sets in English may very 

easily contain pupils at almost all levels of O-grade achievement. 

The effects lOf previous setting can be seen in two of the 'by class' schools. 
Tabla 4.4 gives the correlation coefficients by class for this group of schools^ 
andishqwsithat in schools 32 and 35, especially the former, class teachers did 
a creditable job in ranking their own classes. Indeed, when the class correlation 
coeffideri ts for schbal 32 wereco the appropriate statistical test was 

carried out, it was found ihat theJ^ojio/Critcrion Test correlations were si^i- 
ficantly better thM Jhe O-gra^^^^ ones. The teachers here had, 

then^ ranked their separate classes better than tlje O-grade did, if Criterion Test 
results are taken as the criteribn. This does ript mean, however, that they pro- 
duced a schoo] rank order which was superior: in fact, Table"'4,3 shows that 
there was no-sjgnificant difference between scTioot Foljb and O-grade correlations 
with the Criterion Test,: a^^ school rarik order for school 35, where the 
teachers also fairly successfully ranked their own classes, was actually signifi- 
cantly Ms satisfactory^ one. The reason for this apparent 
contradiction is that the means by which classes had previbusly been set had 
nbt in fact separated pupils int^^^ ability groups. In school 32 the 
top and bottom sections had been fairly well identified, but in the middlegrbups 
there were pupils obtaining all O-grade bands, A to 'F'j in school 35 this hap- 
pened in all the classes for which we have data, except the top section. (Sec 
Table 4.4, 'Notes' columji.) Since classes were, in effect, of mixed ability, it was 
easier ^for teachers to rank: within them than if they had been genuine ability 
groupings: For the same reason, there v/as a large bverlap at both ends between 
class rank brt^ below it ; and this resulted 
in a less satisfactory jc/joo/ correlation with the Criterion than rriight have been 
expected from apparently quite gbbd withiri-class ranking. 

The principal argu in :favour of class teacher assessment as opposed ta 
team assessment is that it is less time-consuming. The researchers believe, how-' 
ever, that the exjra time jre^ui standardising is very well 

spent and that the advantages accruing from it are significant. The very good 
cjass teacher ass^^^ not belie this belief, because the 

teachers in that school did in fact approach the whole pibject Wp^ much as a 
team and there wa^^^^ and discussion, even 

though each carried out his or her own class assessments. 
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: It is possible that class assessments made an important contrLbation to the 
relatively poor 4nterna[ assessments in school 13 (see Table 4.2). TlieFe was 
some evidence from the Markings and the moderation procedure m this 
school that the department had been unable to keep to the instfuctidn to work 
lis a team, each marking the worjc of a cross-section of the year group. 

_ Jj M 'f^W^l^M<^Jl researchers iKat cTass teacher assessrnents should be 

rejected as a means of public vertification because of tire dijficuiries of srandardtsmg 
'^^^'"'J^^^^^'^-^ u'/;r<:/;, though 'present, are more easily overcome for whoTe-schooJ 
assessments. 

id) Some Possible Causes of the Lower Corretatwhs -_ _ 

Before leaviiig^the c<msidc^^^^ of ranking it may be helpful to call to mind 
factors afTecting correlations and,- in particular, to tiy to identify possible 
reasons for the less satisfactoiy performance of schools 13 and 35. 

The correlations between the three sets of asiseissment results could have been 
affected by some or all of six fa^^^^^ ' 

(1) Variability in the perforinan^e of the pupils on different occasions. 

(2) The character of the group of pupils concerned. 

. (3) back of effective discrimination In Folio and the two examinations prior 
to scaling. 

(4) Marker inconsistency. 

(5) The influence of the task set and of choice of tasks, especially in assessing 
interpretation. • 

(6) Organisational difficulties. 

The effects of some of these influences on ail the schools is discussed in detail 
in Chapter Vrr. 

Jhere are no repons for thinking that scfeols 13 and 35_ are different from 
the others with regard to numbers 1-3, except that they did liappen ta^e the 
schools obtaining, respectively, the lowest and highest mean marks in the O-grade 
examination. 

All^e schools probably suffered from^a degreejjf in in the mark- 

ii]i£ {Chapter yil 5how^^ much is likelyj; there, may ha^ been more in- 
consistency in schools 13 and 35 than elsewhere, but, if so^^e concern would 
''^^_tp tiy to find caiise^^ irifluen^jji^, class assessments in 

school 35^ and possibly also in school 13, has alrea^^een mentioned. It Is 
closely related to th^^ 

the tasks set for pupils - in particular, different class teachers probably set more 
and less difficult interpretation tests. If this drawback was combined: with a 
failure to invblye all the staff in the Trial Marking exercises, a school's marks 
would be more vuJnK-able to Inconsistency among the markers than others. 
Teachers in school 35 had, of cou^^^^ been asked to work Independently, though 
they did all take part in the Trial Marking exercises. There were indications ttet 
it was^mbre diffku^ the project in school 13, and, 

although.it is not possible to account for a low correlation solely in terms of 
o rganisati arid ad mini sy atiy e dijficu 1 tj es^ th^ j) rpbably d id pjay a part in 
lowering the Folio/Griterion Test correlation there. Only a proportion of the 
pyp^is cdmplejed the Criteriori Test thefe_be<^use parti of it_were set in class 
and not in the same standard examination conditions as in the other schools. 
A practical difficulty, therefore, donsiderably reduced the valiie of the refererice 
test. TTiere were also irdications in the data that Folio Assessment there may 
have been less co-orditiated than in more successful schools. A* riumber of 
characteristics suggesting *by-class* assessment were noted, though a *whole- 
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school' approach was required. CdmmuriicatiorL within the department may 
have been jdiffieultrin re^ to the qucslionnaire at the end of the project, 

three teachers reported that they had no knowledge of a mbderator's visit to 
the school to discuss the appropriateness for O-grade of work in progress, 
though on';; did take place: 

Comments on OTganisationai problems, which should be linked with the report 
on practical pfoblems in Chapter impressionistic, since it was no 

^ part of the remit of the project to judge the quality of teaching or management 
m schools. It did seem^ however, that a number of the difficulties which were 
metjnjlie course of the experiment in all schools were essentially management 
problems. How to allocate very limited marking time7 How to find time for 
clepartmental dis^^^^^ scheme? How to obtain co-operation from re- 

luctant m^embers of staff? How to maintain folders in good order? And so on. 

For succes^ul Folio Assessment it is protadiy necessarj^ Jo have a member of 
^^P9^W^fl^^ the method and has the time, energy, kwoW" 

Tedge and authority to encourage others, arrange meetings, give advice oh standards 
^'^^^i^ A^^^^^^^ ihsjst on adherence to instructions and the 

full involvement of all in Trial Marking and Moderation procedures. It is, perTiaps, 
a role for a very capable Assistant Principal Teacher. 

JOuQ remaining ikctor characterised school 35 ^ several teachers there entered 
^stjmate\marjcs fcM-i^pup^^^^^ who had failed to complete the minimum require- 
ments of the projector whose performance 'in the Fblip^brk did not match 
the teachers' view of their -ability'. There may have been Some confusion in 
the minds of these teachers as to what exactly they were assessing. 

It would he necessary in a real Folio Assessment . sche/ne id make dhsoTutely 
ctear the rnstrucM^ assess only the work actualiy produced, since estimates 
of potentiality a^e even more likely to be unreliable than judgments about dctuaj 
performance Tn English. 

II. Pass Rates - Cdttiparisbn of Folio Standards with O-grade 
. Standards 

S6hdoIs' accuracy in::assign^ may be judged from a comparison 

of pass rates at each band between Folio Ji^ssessment and O-grade. A similar 
compafison between Folio a GnteriOT Test would have been possible, but 
would have been superfluous as far as consideration oTsrandards is cdncerned: 
the only valid basis for standards available was the awards made by the G-grade 
examiners; so bands A, B, C, D ai.J E on the Criterion Test could have been 
awarded only after scaling its results against the O-grade results. 

The d-grade ani Folio pass rates, before scaling the latter^ are given in Tables 
4,5 and^.6Tor^ 1791 p_uj)ih^ in the II schools for whom both Folio 

and O-grade marks were obtained. 

From the 'Folio {raw)* column can eusiiy be seen the typTcal tendency of in ternal 
assesstnem above, the C pass mark, 

along with the ensuing failure to award enough As, Es and 'F's, 

Raw Folio marks clearly do not produce pass rates at each band cdmpardbTe 
to those of the O-grade exmnndtion. 

The smple compurtson of pass rates set out Tn Table 4:6 is hot, ho we ver, fair, 
dhd FpJio assessm^^^ cannot be condemned on the basis of it, ft is necessary to 
compare Folio with O-grade after the former has been subjected to moderation 
to make it conj[or^ standards. The ^ects of moderation are described 

in the following chapter: 
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TABLE 4.5: OVERALL PASS RATES AT EACH BAND: 1791 PUPILS 
IN 1 1 SCHOOLS 



Bancl 



Folio (raw) 




TABLE 4:6: PASS f^ATES AT EACH BAND, BY SCHOOL 



School 



Bmui \ 



O-firade 



Folio {raw) 



II 

(149 pupils) 



G 
D 



26% 
20% 
27% 
17% 
8% 
3% 



J4% 
V20% 
'* '% 

/ o 
4% 

e% 



13 

(263 pupils) 



A 
B 
C 
D 

E 



17% 
12% 
19% 
14% 
13% 
25% 



8% 
21% 
37% 
19% 
14% 

2% 



L4 , 

(217 pupils) 



A 
B 
C 
D 
E 



24% 
18% 
18% 
15% 
13% 
12% 



5% 
24% 
45% 
20% 
5% 
1% 



15 

(163 pupils) 



A 
B 
C 
D 
E 



20% 
17% 
19% 
19% 
15% 
10% 



20% 
10% 
31% 
30% 
9% 
0% 



21 

(110 pupils) 



A 
B 
C 
D 



27% 
15% 
26% 
22% 
7% 
3% 



25% 
25% 
40% 
8% 
1% 
1% 



22 

(236 pupils) 



A 
B 
C 
D 
E 



27% 
24 %s. 
17% 
17% 
8% 
7% 



35% 
25% 
8% 
4% 



V 23 

(109 pupils) 



A 
B 
C 
D 
E 
'P 




10% 
19% 
29% 
22% 
14% 
6% 
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TABLE 4.6Ccpntd) 



^ School 


BaruJ 


O-grade 


Fotto {jraw) 


- 24 
\ (73 pupils) 


A 
B 

C 
D 

E 


22% 
29% 
15% 
15% • 
4% 
15% 


1% 

23% 
30% 
15% 
19% 
6% - 


32 

(281 papils) 


A 
B 

P . 
E 
'F 


17% 
. 18% 
22% 
17% 
15% 
11% 


14% 
24% 

37% 
16% 
6% 

3% , 


(59 pupils) 


A 
B 
C 
D 
E 
*F 


12% 
9% 
44% 
13% 
17% 
5% 


3% 
19% 
53%. 
23% 

2%. 

0% 


35 

(131 pupils) 

\, 


A 
B 
C 
D 
E 


34% 
15% 
24% 

15% ! 
8% i 

4-%. - -..-4 


18% 
24% 
32% 
24% 
2% 
0% 
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CHAPTER V 



MODERATION 

Three types of moderation device were employed^ 

I Trial Maricing' by all the schools of sample scripts provided to try to 
establish cqmihbri staridatds. 

II Marking: of a representative sample of each school's folios by external 
moderators. 

III _ Seeing oT each sch(y^^ folio assessments against its performance in the 

O-grade examinations. 

ziThe Junction of moderation iithe SCRE scheme: was to ensure that internal 
pass rates were-as near as possible to those the schools would have obtained 
in the external: examination and to ensure as much fairness to every individual 
candidate as possible. 

Of the three approaches to moderation attempted, the «flfects ofithe visiting 
moderator scheme anjci^of scaling against O-grade performance could be gauged 
from the statistical evidence, h^t that of the Trial Marking exercises cpiydi^nbti 
l^achers' respbri however, and the evidence from oSier 

exam boards of their value as in-service training' indicate that they shbuld be 
included in any internal assessment scheme. 

Appeniyces 5 and 6 COT both trial Marking 

arrangements and the Visiting^, Moderator scheme. The concent here is only 
with the effectiveness of the Visitirig^Moderatbr arid of Scaling for ensuring 
cbiriparability betweeri Folio and O-grade. 

I. Visiting Moderator 

Apart rrbm school 33, where judgments aBout the sample folios for moderation 
were unreliable because several bf the folders supi>lied were jncd all the 

sehMls passed the three moderatiop tests (which were those developed for CSE, 
described_in :the Schools Council Examinatii.*ii Bulletin I^o^5:_arid_used by 
TWYLRBB (T3ie _^sf Yorkshire and Lindsey Re^onal Examination Board): 
The teachers and the moderators were m reasdnabTe Agreement as tb ranking, 
standards and the discrimiriatibris made ariibrig the'pupils. 

j^f^ J^f 1^^<>P^ fL^ Y^^f€_ therefore yin^^^^^^ visiting moderator^, md 

the discrepancy in pass rates (see Tables 4:5 and 4\6) was not corrected by the 
moderators. 

The discrepancy In piass rates was caused by three factors which the three 
riipderatibri tests airii to check: 

1. disagreemrat on the sizeif the dis^^ 

2. disagreement on standard,: or excessive severity of leniency^ the Schbbis 
Qouncil test a<x as_saysfactbiy discr^aricies betwe^ moderator and 
school of up to half a j^ade, oa^ in Scottish terms, one range; 

3. disagreement on ranking, which, if jt iis significant and consistent, probably 
means that irioderatbr arid teachers are valuing different things in the 
scripts. , . . 

fn which of the three areas did the moderators' failure occur? 

' Sec, c.g. Cohen and Deale, (Schools Council. 1977). and Chapter X of this report. 
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1. Discrimination: faiiure to overcome iunchtng' 

"^he failure of the modera^^ surprising. Firstly^ the 

sjzeiof a sample which can be remarked in a realistic time makes bunching 
harder forthe schodl to illustrate. ITie schbo^^^^^ the sample 

sol thatithe dumber of moderated pupils in each band was proportionate to the 
whole school numbers in each band, but this may in fact hjiye been top time- 
caHSuiri|hg:^an^inc to be kept to. Secondly, and probably more sig- 

I nificantiy, both teachers and moderators were markirigjvith the O-g-ade stan- 
dardised marks pattern 70plus=A, 60 pIus=B, ^d so on. Markers 
of the O-grade papers also mark with this categofisatidn in view, anc they'db 
^"J?'^^ prociuc^^ that obtained in the Folio 
Assessment (as can be seen from the superimposition of the histogram of the 

' Folio distribution of raw rfiarks bri that of the C^ade distribution of-raw marks, 
which is contained in AppeSdix 16). A standard deviation of about 12-5 is 
normal for O-grade English raw marks, arid that fprthejupils involved in the 
pre^ect^w^s actuaUy 11-78 of this sortmeans that 3 or 3-5 raw 

marks are equivalent to one Q-grade range, and 6 or 7 raw marks to one band. 

is riot the p-^rade Engish markers and examiners, nor, therefore, the 
moderators of an interaal. assessment scheme who effect the appaferitly clear 
discrimiriatiori of pUpill into baridsiAi to -F, but the statistical standardising 
r procedure, which stretches, out raw mark4:anges45f 3 or 3-5 rnarks arid barids 
'i of 6 or 7 to staridafdised fariges of 5 mar^^d barids of 10. Most of the dis- 
. erepancy in pass rates between Folio and ^gcade in Table 4.5 is due to the fact 
that raw JFolio marks have been cbriipared ^i|h staridardjsed O-j^^ results, 
ari<i JhejdJRererices are principally in the distriBption of marks rather than in 
the means: I : 

It iriight be thbuj^t that gives O-grade English 

results a spurious appearance of fine discrimination ; and, NVheri iriter-marker 
incpnsi^tericy is cbrisidered (see GhaptCir of the discriminations 

m^de by the markers is in rather more doubt than is normally the case in the 
. mind of tJie public, including teachers arid marjcers themselves. Since small errors 
iji assignihg pupils to their proper places in the rank order ms^^ ie exaggerated 
by the standardising procedure, some individual pUpils may obtain awardsigiiite 
difTefeht TrK3^^^ (The effects- on individuaj-pupils 

of scaling Folio marks j^re discussed below, pages 44-49.) 

// is ctear that, iflhe aifn is to obtain Foljo aVvqrds distributed in tJie same way 
as O-grade awards, some form of the S€E£B siatisticai standardising procedure 
^^^^ ^'^^^^^^^ 

for [his purpp^^^ procedures, however, impose an appearance of 

finer discrimination among candidates than is actually achieved hy the markers. 

It would be possible for ii Visiting Moderator to check the severity or ieriiericy 
of schools' raw marks, leaving the Stan of marks to be 

eRected by the same procedure which ^t present applies to the O-grade marks, 
vifU Jhe distribution of aw^^^ so that a 

gLi^^nitandard deviatiori is obtained, and the standard deviation for each school 
is then calculated. 

Did the Vfsiting Moderator scheme in the SCRE pfbject in fact show that it 
cbuld satisfactorily correct severity and leniency? 

t^'"?^^^^^^^^^*'^'^^ P-grade examiriatiori, the difference 

between the Folio raw marks mean and the Folio mean after scaling against the - 
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b^grade marks can be taken as a measure of eiich sclibdrs severity or leniency 
relative to the O-grade. These differences were: 



School J J 


+ 0 95 


Raw Folio severe 


School 13 


-4-57 


Raw Folio lenient 


School 14 


^ -0-68 


Raw Folio lenient 


School 15 


* -2-65 


Raw Folio lenient 


School 2 J 


- 1-82 


Raw Folio lenient 


School 22 


F6-50 


Raw Folio severe 


Sehool 23 




(beyond normally acceptable limits) 


-0-40 


Raw Foiio lenient 


School 24 


• r 4-05 


Raw Foiio severe 


School 32 


-3-51 


Raw Foiio lenient 


School 33 


-2*34 


Raw Folio lenient 


School 35 


1 3-90 


Rav Folio severe 



While there miay be statistically significant differences between means at the 
955i^ confidence Hmits. it is normally taken that differences smaller than half 
a grade in eSE terms, or ^ne range in SCEEB terms, are not educationally 
J^eaningful^ A jjiffere^^ which is significant at the 99 % 

confidence limits, would mean that a school had marked severely or leniently 
^h^^ A^n^^^ Pijopprtion oT its ca^ had received awards 

the same as those which the O-grade was making to other pupils whose work 
\ was of a standard which^ exarniners^cGuld re^^^^ as bejnj elear!)^ different, 
NAccordirigly, the moderation test employed accepted as satisfactory differences 
fcetween the moderator's mean and that of the school of up to brie range, or 
5 percentage marks. . 

jHi^act the test fGr severity/leriiericy was passed by all the schools, except 
school 33. When the schools* ravt' and scaled means are compared, however, 
it JS fbiind that school 33, thoughilenient^^^ within the acceptable limits, 

while scfiool 22, where the school assessments had been vindjcated by the 
moderator, was, in fact,, uriacceptably sever^^ 

Tfi additlbif tb^niese^^^ failures, the SCRE mbderatioh sufTered Trpm the 
epnfiisipn about the real size of 'one ranged which was referred to above in 
/the discussjon of dijy:ribution of marks. While teachers, O-grade markers^ ^d " 
inQderator* rnad the idea in mind that one range equals 

five marks, they actually bunched the distribution so much that, in reality, the 
size bf a range was abbut three or thre^^ By this criterion 

schools 13, 24, 35 and (just) 32 were also severe or lenient by more than one 
though the^m^^ ^ests ^ere riot designed to .gu^^^ as fine 

a degree of accuracy as woiild be necessaiy to check discrepancies of only three 
murks - tedeed, it is very doubtful if discreparicies of that order are significant 
educationally (as opposed to Statistically). 

1 /i'l'-y f^^r /to/ (he actual narrow distribution af raw percentage marks, com- 
bined with tfw beiiefon tie part ofreucfrers, markers, and moder^^^^^ that ranges 
^^f"^ Mder than M^y r^aOy Mre, causes serious difficulties for effective moderation 
of standard as well as of spread of mark's. 

The SCRE moderators, despite their experience us O-grade markers and their 
having smisfa^^^^ own Trial Marking, were 

hot able to correct severity or leniency adequateiy. 

VFpr cpnfirma^^^^ asserUons^ see Scho^Js Cijjincil l^an^iinatio^ Bulletin 36, •Mode 

Comparability in the CSE*, by Bloomfield, Dobby and Duckworth. (1977) 
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^ TWs /uilure may have been due to their^ inevitable lack of experience as 
moderators, to the pressure to complete iiriSmtliar work in a short time, or to 
the unreprese_niativ_eness of the schools- sampie scripts: The task is made harder 
bjrcause the bunching of marks and suB^uerit standardising cause the published 
O-grade results to make discriminationi among pupils (obtaining contiguous 
^ ranges which are too fine to expect moderators to make tgem reliably. 

The conclusion of the researchers is that the SCRE Ynodergtion Was^^ n^^^^^ 
cessfui enough to ensure cqnifyi^pmity of O-grade EngMi standards airjoss sdivois, 
hut yvith the significant /jote thai reiizibie ctassificatlon into ranges of 3 or 3-5 raw 
marks w idth is almost impossible in English. 

^ f>JsiJgresme/u on ranking 

The TWYLREB test for 'conformity', or; In effect, agreement on ranking, is 
desijgned m auch a way that, if it is passed, tfeeirbrrelation between the school 
marks arid the m^^ for 20 sample scripts is not less^ than about -70. 

(See Schools Council, 1965, page 25.) This does riot mean, of course, that the 
correlation for the riiode^^^^ and the teachers' marks for the whole school 
would be the same, bat it should mean j hat schools passirig the test are in 
reasonable agreement otv ranking \yith the external examination, for which, in 
the case of theJSCRE project,: the moderator was a possible substitute, in fact, 
although all schools passed thjs test, itihas alrc^ been shown in Tables 4:2 
arid 4.3, that schools 13 and 35 obtained Folio/b-grade correlation coefficierits 
?^ '^^J'^spec^^^^^ normally be regarded as 

iridicatirig acceptable comparability of ranking between two assessments. 

. 7?'^' Moderator procedure therefore jMed ufso to identify the two 

internai assex^nwnts which were relatively unsatisfactory with regard to rankiitg. 

IL Sealing 

Folio marks might have been scaled against raw O-grade riiarks so thatHCach 
school's mean and distribution of marks was the same as those it obtained in 
the ^O-grade before standardisation of the latter. It would then have beeri 
necessary to apply the SCEEB staridardisirig procedure to the whole group of 
Folio pupils, in the sanie way as it is applied to the whole population of O-grade 
candidates. The method employed by the researchers was, however, to app 
scaling arid a fonri of standardising to the Folio marks simultaneously by scaling 
them against the O-grade standardised marks. 

Leaving aside tfie possible disadvaniages for the accuracy of individuarpuplls' 
awards, it is clear from Tables 5.1 and 5.2 that pass rajcs at each band were made 
much more cpniparable by scaling Folio marks against O-grade standardised niarks 
{though a .flight bulge still remained in Folio bands C and D). 

TABLE! 5. j : QVERALLJ>AS5 RATES AT HACH BAND: 1791 PUPILS 
IN II SCHOOLS 



Baml 

A 
B 
C 

b 

E 



O'graae 

22-3% 
17-4% 
2J-4% 
16-4% 
li-6% 
10-9 "r. 



- ! Folio 

Folio {ra w) | (scaleei agaittsi O-grade ) 

12-2% ' 2V'7% 
20-4% 45-4% 

37'4% i 22-8% 
20-3% 20-4% 
7-6% 10-0% 
2^1% 9-7% 
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TABLE 5.2: PASS RATES AT EACH BAND, BY SCHOOL 



ScTiodJ 


Band 


O-grade 


^Foiio {raw) 


1 

_j_ Folio {scaled) 










11 » 
(149 pupils) 


A 
B 

C 
D 

- 1- 
•F 

- i 


26% 
20% 
27% 
17% 

3% 


■ 14% 

: 20% 

44% 
18% 

; 4% 
i 0% 


! 22% 
) 19%' 

44%* 

12% 

6% 

i 

! ♦Note: 

\ Scaling. had hd 
effect bh the^ 
number of C 
awards because 
Folio 

Assessfncnl 
j was made in 
ranges, not marks: 


13 

{Zv)J pUpiIS/ 


c 

D 

' ; 


17% 

12/^ 

19%' 

14% 

13% 

25%- 


' 8% . 

2 1 % 
37% 
19% 
14% 
2% 


i 17% 
11% 
17% 
22% 
12% 
21% 


14 

{zi 1 pupiis) 


' A \ 

D 
D 

c ; i 

D ! 
E 

•F' \ 


24% 
Io% 
18% 
15% 
13% 
^2% 


5% 
24% 
45% 

i 20% 
' 5% 
1% 


20% . 

15% 

22% 

22% 

10% 

11% 












----- 15 

( loJ pupils) 


A 
B 
C 
D 
E 
•F' 


20% 
17% 
19% 
19% 
15% 
10% 


20% 
10% 
31% 
30% 
9% 
0% 


21% 
9% 
•21% 
22% 
2f% 
6% 


21 

(1 10 pupils) 


■ A :. 
B 
C 
D 
E 

'F* 


t5% 
26% 
22% 
7% 
3% 


25 % 
25% 
40% 
8% 
1% 
1% 


26% 
14% 
30% 
23% 
5% 
2% 


22 

(236 pupils) 


A 
B 
C 
D 
E 


27% 
24% 
17% 
17% 

8% : 

70/ 

'0 


11% 
16% 
35% 
25% 
8% 
4% 


27% 
24% 
22% 
16% 
6% 
5% 






23 

(109 pupils) 


A 
B 

C i 

D 

E 
.p. 


• 19% 
14% 
24% 

15% 1 

14% ; 

14% 


10% 
19% 
29% 
22% 
14% 
6% 


17% 
17% 
19% 
24% 
6% 
16% 


24 

(73 paptis) 


A 1 
B 

e ! 

E 


22% 
29% 
15% 
15% 

4% 
J5% 1 


7% 
23% . 
30% 
15% 
19% 




26% 

26% 

10% 

12% 

14% . 

12% 
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TABL E 5.2 (cbhtdj 



ERIC 



Schooi 



32 

(281 pupils) 



33 

(59 pupiis) 



A 
B 

e 

D 
E 



35 

(131 pupils) 



A 
B 
C 
D 
E 



A 
B 
C 
D 
E 



O'grdde 



Folio (raw) 



Folio (scaleci) 



17% 
18% 
22% 
17% 
15% 
11% 



14% 
24% 
37% 
16% 
6% 
3% 



19% 
15% 
21% 
24% 
10% 
11% 



12% 
-9% 
44% 
13% 
17% 
5% 



3% 
19% 
53% 
23% 
2% 
0% 



34% 
15% 
24% 
15% 
8% 
4% 



18% 
24% 
32% 
24% 
2% 
0% 



12% 
17% 
29% 
25% 
14% 
3% 



34% 
34% 
22% 
20% 
8% 
2% 



(J?) Effect on mdivTduat awar^^^ 

Er^ad^ speaking, therefore, scaling corrected the mean and distribution of 
internal assessment marks to match the Orgrade ones. What were its effects on 
individual school pupils? 

it is heipfal to consider the effect of the discreparicy in pass rates at each 
band before and afteF scaling on the *match' between Folio and O-grade awards, 
i.e., on the numbers of pupils receiving the same range, plus or minus one range, 
etc., in the two assessments. 

Tables 5.3 and 5.4 give the overall figures before and after scaiing.' 

TABLE 5.3: *MATCH* OF AWARDS: FQttO (RAW) - O-GRADE 
(1791 PUPILS IN 11 SCHOOLS) _ _ 

JVo. of Ranges 
lytisptaced 



-8 -7 -6 -5-4-3 -2 -i i 0 



3 ; 4 ; 5 , 6 7 8 1 9 



Freqnency 



1 3 16 58 68 L12 221 264 , 289 j 240 , 21 1 [149 84 t 34 ' 14 4 i 2 I 



I % 3 % 4 % 7 % 1 2 % 1 5 % 1 6 %!l3%:i2%;8^|5 % 2 %: 1 % 



Folio more lenient 



Folio more severe 



SaiTie range — 16% 

Wiihin ±1 rarige = 44% 

-i;2 ranges = 68% 

;h3 ranges = 83% 



(pcrecnlages arc approximate) 



Assessed more 

severely by Folio ~ 4j % 

Assessed iribre 

leniently by Folio = 42% 

'Appendix 12 contains figures for 



^hool separately. 
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TABLE 5.4: MATCH' OF AWARDS: FOUO (SCALED AGAINST O-GRADE) - 
0-GRADE(l79l PUPILS IN W SCHOOLS) 



mjs- _ 
placed 



No^Qf . : . L _ . _ _ _ : 

ranges -10-9 - 8-7-6-5-4-3-2 — 1 0 I 2 3 i 4 5 6 7 8 9 ;jO 

mis- ^ ' ' 





Pre- ■ ' ; ■ t i ■ / ^ | l i J ' C - ' ■ 

quency I 14 19 1 17 !44 { 75 |128;I75 j25I j 377 i 266 I 176^123169 46 ; 20 ,4 3 0 2 




Folio more lenient 



Folio more severe 



SjMTiexange L I — 21% 
Within ± I range: ^ 50% 
Within ±2 ranges =70% 



(percentages arc apprbxtmatej 



A_ssesscd_rnorc _ _ : 

severely by Folio 30% 

Assessed more 

leniently by Folio ==39% 



Scaling agaimt O-grade ^ixmdardised marks tmproved the percehjdge of pupils 
witfjin ±1 range dy 6% {to 50%) and that af pupils within +2 ranges by 2% 
{to70%V " ^ ^ 

Some extreme discrepancies between Folio and<)-grade ranges were, however, 
exaggeriated by: the' scaling process, so that the: greatest differences between 
Folio and O-grade ranges after scaling became 4-10 (2 cases) and - 10 (I case), 
whereas they had previously been [-9 and -8 (1 case each). 

When each school wasi considered thdivtdually7tt was diicirifiered jMr tT^ match 
^f ^^W^^M f'dn^^^ was not always improved by the 

scaling » and in some cases was less good after it. There were also several cases 
^M^'^S ^^'^ schools where the greatest discrepdncy was larger after scaling than 
before. 



» A Note on'Schddl Examiiratioas 

An i nc ident^I but _notable_point is _thaL t ha^sercentages: of pupils obtai ni hg the same award 
within ^ J and 2 rahgw_on^Folio^dO-gradc reported in Tabl&5:4 may be coinpsU-ed witH 
those reported by SCEra \xiAnImestigatianJato7he_€ompmGktav of School EstTmates and 
ExMninaUqn Perfo were matched 

afi?i»nst O-grade marks. TTie j)erccntagc of pupils assigned the same range tjien (before 
«^mg was_19%;^49% we^^^ ± I ran^; and_7i% within ±: 2 rangs.:aa compared 

wi^n \^/oj ^A^^ assessment. It is not possible io conclude ifrom these 

"Syres^^n^^ school cxannuinat|o_ns are beuer than Folio Assessment l?y_comparisott the 
O-gradfr examination, since there may ^significant differencesjbetween lhc_character of the 
two study populations, the SCEEB investiifation having been carried oiiti)eforellieraising:Qf 
the school leaving age had ehcdura^ many less ablej)upils to take the_examinallon. School 
cxarninatibxis are. howeveTi rnore fake the O-grade thanl Folio assessment^ and this is^tru^^in 
particular of the testings of ihterpretatloh, which the SCRE pr(^cct found to be unsaLisfactory 
in FoUo A^essment. If iriterhal assessrhent were intended on^^ to diQ)licate the funclion of 
S5fS5?^- ^f?'"'??^'^'^* ^ school exaininatibh would certainly be as satisfactory ^ means of 
Itniunng the requirement as Folio Asscssrrient. given adequate Trial Marking* experience for 
- the teachers and some moderation of the level of difficulty of the examination set. 



45 



The foljawmg^^^ tihows the percentage of pupils in caich school within I I 
range on Folio andO-grade before and after scaling of the former. It Msa shows 
^^V^^^cct of scaling on the deviation of each school's Folio 

marks: The amount of discrepancy remaining after scaling is that due to ihe 
^"^"^?ce ofthe VaribUs^^^^^ rankingisuch as variabte performance 

by the pupils and unreUability in botii FoJio and O-grade: the correlation cb- 
etlkients given m the table for each schoGl, tjiqu^h ciilculated for very slightly 
different numbers, are indications of the amount of agreement on ranking 
between Folio Assessment and O-grade, sb that it is possible to see approxi- 
miitely hpw many pu^ each school received different awards (after scaling) 
. in Folio and O-grade, when the cbrrelations were of given sizes. 

TABLF. 5.5: EFFECTS OF SCALING FOLIO ASSESSMENT AGAINST 
O-GRADE MARKS (BY SCHOOL) 



School Folio Raw Folio Scaled Kaw FoUo Seated Fdtid r-FoUa/ 

match mrh ftiaJch with O-grade 
: MHAN SD MEAN SD O-grade O-grade 

I l_ 5710 ll-Iyl 5815 16-24 , ie I range. 48-4% \ I ran-e 42-8% ' 68 
( H? 3: 2 ranges 71-8% ±2 ranges 68-3 % 1 (calculated for 
P^Plis) ,143 pupils) 


13 , 52 90 : j2-24 : 47-43 ' 22 j5 ^ i. i range 37-6% 
<262) , ±2 ranees 58-9% 


±1 range: 42-6% 
±2 ranges 59-7% 


' '11 
1 (262) 


14: ; 54-24 ' 9-79 • 53-56 i 19-35 
(212) 1 . 


± \ range 41-0% 
±2 ranges 68-6% 


±i range. 502 % 
±2ranges7I-4% 


! -!74_ 
1 (196) 


IS 55-39 1 14-26 1 52-74 ' 18-75 1 ±\ range 45-2% 
(163) ^ - -- -x 2 ranges 7 1*4% 


±i range 45:1% | -81 
i2 ranges 67-7% ^ (154) 


21 ■ 60*90 ! 10-98 ! 58 08 ' 15-72 ( ± 1 range 57-6 % 
(110) - J:2 ranges 84-6% 


± \ range 64-8% [ -83 
±2 ranges 82-8% (93) 


22 ; 52-61 ' 13-28 ' 59-11 i 18-87 ! ± 4 range 4P2% 
(236) j d:2 ranges 65-4% 


±\ range 54-7% 
±2 ranges 76-1% 


•79 
(204) 


^.. 23 ! 52-14 I 14-41 1 51-74 i 20-04 t J:: a range 58-8?^: 
(109) i i : 1 1 i2rang^75-3% 


. ±4 range 58-4% 
±2 ranges 74-5% 


•79 - 
(81) 


24 ! 51-60 1 13-75 1 55-75 | 19-70 ! -h i range 54-8% 
(73) 1 i :i:2 ranges 75-3% 


ih 4 range 46-5% 
i 2 ranges 67-0% 


(70) 


32 j 55-90 1 n-94 52-39 ' 18-32 - 1 range 40-9% 
(281) } 1 i 2 ranges 68-7% 


± X range 49-4% 
± 2 ranges 66*9% 


•71 
(261) 


33 • f 54-37 ' 8-79 ' 52-03 ' 14-58 ' I range 44*0% 
__L59) . :-: 2 ranges 72-8% 


hi range 44-4% 
-12 ranges 62*8% j 


•39 
(45) 


35 57-48 10*98 6j-38 , i8-62 i 
(131) ; ! 

1 ■ ! ' ■ 


±i range 39-7% 1 
-b2 ranges 57-3% ' 


±4 range: 42-0% 
±2 ranges 63 0% 


!59 
(112) 



This table, containinj^ information of several different kinds presents some 
problems of interpretation : no obyidus pattern strikes eye. It js possible, 
howeveFi Jo iriote a number of points for consideration in trying to judge Jn 
what conditions scaling of Folio Assessment against standardised 0-grad_e marks 
is likely to incre^^ of pupils given the same or nearly the same 

ranges in both assessments: 



.1. The figures in the coiumii headed *ScaIed Folio match with O-grade' 
indicate the amount of mismatch due to disagreements on ranking. 
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In scliooJs obtainiiig Folii) O-graycjeorFchitions of the normaljy acceptable 
orider ofi-70 or better, after scaling has removed the cflccts of the schobjs* 
A^o^% leniency and of their idibsyrieratic distributions of marks, up to 
40 /„ of the pupils still obtained marks in one assessment which difTered by 
two ranges (or one fall band) or more frbitl those obtained in the other. 

2. The scaling process caused the match of ranges in some schools to be 
worse than before (see schools j I, 15, 23, 33; and also 21 and 32, where it was 
improved for ± I range but was worse for ± 2 ranges). 

Distinct improvements In the match occurred in schools 13, 14, 22 and 35. 
Is It possible to identify any characteristics common to .schools where the match 
was improved? The following sections seek to answer thi.s question. 

. 3. What is the effect on the match of ranges if the scaling process causes a 
large shift at the mean? 

The largesl mean differences occurred in schools 22 16-5 marks) 13 (4-57 
marks), 24 (4^^^^^ marks). In threeiof these cases there was 

indeed an improvement ia the match, and in sdlbol 22 it \va_s a quite marked 
improvement. In school 24, hbv^veri the number of pupils obtaining the same 
or nearly the same range in Folio and O-grade was after scaling. It 

may also be noted abbUt thbs^ schools where the shift at the jnean was no 
more than about 2'5 marks at mo5t - schools II, 14, 21, 23 arid 33 - that all 
(^cept school [4 suffered a drbp.after scaling in the number of pupilsvobtaining 
the sarne_pr:: nearly the same range in both assessments. It appears, theri, that 
when scahng corrected a noticeable ariiburit of severity or leniency at the mean 
It generally lediajso to an increase in the numbers of pupils given the sanle 
awards by Folio and Q-grade. Certain bther factors_must, however, have been 
influential in schpp_r24 ahd may also have contributed to the poorer range 
matching after scaling in ihose schools which had nbt reiquired large adjustments 
to correct severity br leriiericy. 



.^1 4^: One of the other critical factors influencing the match bf fariges awarded 
-IS the effect of scaling brt the dispersibrii of marks^ which -can be judged by 
comparing the raw and scaled standard deviations of each school in Table 5.5. 

The sizcjof tjie acQustment^^^ scaling to the dispersion of marks seems 
to be of some significance to the match bf rariges, especially jf a large adjustment 
of dispersiqri is combined with a fairly large shift at the mean. In schools 13 
and 35 there were shifts at -the mean of some fbUr br five marks and a much 
wider dispersibrt bf marks alMs^^^^ with improvements in range 

matching (though only a small improvement in schbbi 13). The difTerence'" 
between raw and scaled standard deVjatJo^ two schools is of the order 

pfjI-IilLJt was only about 4-6 ih allthe schools which had pobrer range riiatching 
after scaling.. This means that the latter schbols had sjread out the pupils more 
effectively than the 1^ raw Folio Assessment, and had, for instarice, 

awarded more band As. Scaling will have mbved sbrii_e pupils within band A 
to other rari^es,^J^ range 5 to range 3, range 4 to range i, so increasing the 
count of pupils misplaced by one or mbre ranges. By the criterion lof accurate 
^ssigftment bf rpges^^therefore^ schools using a wider spread of marks in Folio 
Assessment may appear to be less good judges of standard thari others^ jf no 
attention |s paid tb the ability levels at which ranges in Folio and 0-gradc 
are matched. 

Aj:pmparison between the effectSL of scaling at three ability levels iri schools 
14 and 15 will illustrate this pbirit. Scalirig made small differences to the means 
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in each scKool, but a much greater diflerence to dispersioh in school J4 than 
in school 1 5, so that in school 14 pass fates at bands A and 'F* were very different 
afterwards. Table 5,6 shows that in school 14_scaling brought drainaUc increases 
in numbers of pupils obtaining bands A and 'F^ in both Folio i^d O-grade, but 
not in school [5, wh^ had already made quite a good job of 

assigning pupils to the extreme bands: 



TABLE 5.6: SCHOOLS 14 AND 15: EFFECTS OF SCALING AT 
THREE ABILITY LEVELS 



S^chool 

"mT i 



Nttmberii ohtainmg Folio Bands 
before and after scaling 



O-^radc hands acntady obtatned 
by iJiese pupUs 



15 





Vass Rate 




B 


c 


D 


E 




Folio 'A* Raw 


- 11 


-5-4%~ 


-7 


-3 


i 


0 


0 


0 


Folio "A* Scaled 


- 43 


19-8% 


26 


12 




0 


0 


0 


Folio :C' Raw 


: 97 


-^44-7% 


12 


21 


28 


18 


11 


7 


Folio 'C* Scaled 


' - 48 


- 22-1 % 


4 


16 


13 


I 


4 


2 


Folio 'F* Raw 


- :3 


- 1-4% . 


Q 


0 


0 


0 


1 


2 


Folio ' F' Scaled 


■ : 24 


111% 


0 


1 


1 


2 


7 


13 


Folio 'A* Raw _ 


- 32 


- 19-6% 


24 


7 


0 


1 


0 


b 


Folio *A' Scaled 


^ 34 


- 20-1% 


24 


H 


0 


2 


0 


b 


Folio *C' Rawl 


- 50 


-37-7% 


1 


12 


17 


13 


3 


4 


Folio *C' Scaled 


- 34 


-20-1% 


2 


9 


lb 


11 


1 


1 


Folio *F* Raw 


l: 15 


- 9:2% 


b 


b 


3 


1 


7 


4 


Folio *F* Scaled 


11 


-6-7% 


b 


b 


2 


j 


4 


4 



Table 5.6 also illustrates rhe typical effect of the scaling procedure on average 
pupils. In schools 14 arid 15 arid ill all the others the number of pupijs obtaining 
band G in both Folio and O-grade was smaller after .scaling than before. 

The main effect of scaTutg was to increase the match at the extreme hands and 
decrease it tn the middle of the curve. 



5. The effects of scaling on individual pupils in schools 14 and 15 do not 
depend solely on differences of mean arid dispersion. The ^greemeniio^ 
between Folio and O-grade alsoJias a significant influence: The respective cor- 
relation coefRcients (scliopj 14: -74 for i[96Hpiipils; school 15: ^tfor 154 pupils) 
are' not significantly different. Botlr correlations would be accepted as sati.s- 
f ^ctpry between two assessn of Eriglish» yet scalirtg had d ifferertt effects bri 
individual pupils in the two schools. It is clear that a high correlation between 
Folio arid O-grade riiarks does riot necessarily mean that scaling will be effective. 
Correlation coefficients riiay hide other important factors. The reader is referrjed 
to Appendix 8, which contains an examination o£the results of the two schools 
showing the most favourable arid unfaypurable ef^^^ 

some way to defining the circumstances in which the scaling procedure applied 
is helpful. School 22 showed the gre^alestiricrea^ scalirig: 
school 24 suffered from more di.screpancies after than before. 

Ist brief, the stud^ of these t\vo schools shows tlia.t, though both have res- 
pectably higli correiations between Folio and O-grade and between Folio and 
Criterion Test, riiisiriatch in farikirig between Folio and the other measures is 
distributed in different ways. In school 24 one class in particular is badly ranked 

48 



50 



by cqmparisqri with t^^ CHtenpn Test and another of lower attaiiiment is: ex- 
tremely well ranked and strongly influences the overall cdrfelatibri • in additLorii 
the former has been assessed renjentjy and the latter a little severely. The effect 
of: scaling was to exaggerate the ranking disagreements in the poorly ranked 
class, and to cause some injustice to the p^^^ other class* because the 

scalmg prpcessi in trying tb counterbalance the leniency shown to the better 
class, dealt^withithe less able one even more severely. (The details can be seen 
oil pages 176-177.) 

School 22, pn jhe other hand, had been approximately equally severe at all 
ability levels and had also maintained a consistent 4egre_e of agreement/dis- 
agree^ment on ra^^^ Scaling, therefore, had a very beneficial 

influence, adjusting the overall severity feirly and pushing the best pupils up 
to the top marks and the ppbre^^ to the lowest. 

J7te scaling procedure will cause some inuccurate awards id TndivTdudl pupils 
uniess ihe agreement in ranking Js ^dhsis tent across the school, i,e„ unless liiere 
are ho special 'class effects* in the School rank order, 

III. What Sort of Reference T^t ? 

Since, in the SCR E project, FoliO: Assessment and O-grade were designed to 
test the same curriculum, the difflcalties caused for the refePence test method 
by variability of curricula did not arise 4ri aj noticeable vvay. 
•1 JfLSC^hpbJsj took the oppor^^^ of developing courses notxilosely related to 
the requirements of the external examination, the problem of the validity of a 
reference tesLfpr scaling internal assessments would be critical. It can be seen 
however, from the correlation coefficients tabled in Appendix 13 that all the Ave 
sepaj-ate elements of the Criterion Test measured achievements also measured| by 
the Folio Assessment : a reference test containing, e.g., the two or three Criterion 
Test elemeirts cdf relating mq^^^^ the Criterion Jest as a whole wouid 

probably cover xnost of the areas^ of written work Enjlish teachers would wish 
to engage in. (They would not, hpweyeri be suitable reference tests for, e:g , 
pra_l:pr<irama work.) It can be shown in fact that combinations of ah£ Jwo 
Cntenon Test elements in the SCRE j>rcject:^produce at worst a correlation of 
•66 withjhe Folio Assessments (for a total of 1529 pupils), Combmatidn of a 
-writing task (either Criterion Test Paper lA or Criterion T^st Paper III) with 
Cf itendri Test Paper JHi(trad^^ ititerpretation) or Criterion Test Paper IV 
(multiple choice interpretation) leads to correlations betweeajhe combined 
marks and the Fblip^Tp^Lbf abbut-70. Table 5.7 shows the correlations be- 
tween particular elements in the Criterion Tes^ and the Fdlib total mark and ' 
between combined elements bf the. Criterion Test and the Folio total; Any of 
iMcpmhinations of papers shown except that of Criterion Test Paper III (speci- 
fied writing tasks) Avith Paper IV (multiple choice test) can in general be said 
tb hav^e ineasu of the same qualities as the Folio Assessment did to 

make their use as reference tests justifiable, arid iy s Hkely that the addition of a 
literature test to the_ Paper mpl^ IV combination would bring its cor- 

relation with Folio total to over -70: Two points shbUld; however, be remembered. 

1. These correlations are for a large number bf pUpils: sbriie individual 
schools would achieve less satisfactbiy bnes. 

- 2. -The calculatibri* bf the papers and the 

FpHb Jp^Lmatheniatically determines 'weighting coefficients* br niUltiplierSi to 
be applied to the marks in each paper in brder to maximise the correlation of 




Sec Q. McNemar (196?.). page 175. 
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the combined papers with the Folio Total: These 'weighting cbefiiciehts' are 
given (approximately) in th^ headed, having been calculated by the 

method given by Q. McNemar (iS62)z fn'-jlevising a reference test including 
two tasks like those of different Criterion Test_papersj it would be necessary 
to give appropriate attention to the relative weightings of the two tasks in order 
to achieve the maximum correlation between the reference test and the internal 
assessment The - we^^ obtained herb may give a lead in trying 

to determine what weight should be given to different test elements when Com- 
bined with various othe^^^^ would be absolutely appropriate only 
in the case where the correlations between internal assessment and tfee two 
separate reference test elerne^^ identical to those obtained in the project 
for Folio against each relevant Criterion Test element. 



TABLE 5.7: CORRELATIONS BETWEEN FOLIO AND CRITERIOIS TEST 
ELEMENTS WHICH MIGHT FORM REFERENCE TESTS 



CriJeno/i Tesi 
elemems 


UorreiatJon with 
Folio Total 


Qo^fficiefxts k^^^ 
\ combined papers 


Paper 1A-: 

free composition • 


•61 


i 

1 


PapetJIIr^ : ! 

specified writing tasks I 


-59 


j : 


Pajjer lA and B - ; 
free composition and literature ; 


•69 




Paper II - 

tradittbnaj interpretation 


•66 




Paper iV- 

multiple choice interpretation 


•58 


i 


PajDers lA arid 11 - | 
free cbmpbsitiori — plus 
tniditibrial ihterprelati bh ; 


•715 


1 Paper lA = 1-5 
j Paper 11 ^. 1 

1 


Papers lU 4-11 ' 
(sp<xific writing tasks jj) us 
traditional ihterpretatibh) 


•709 


i Paper III - 15 
; Paper II ^ \ 


Papcre lA -f IV - 
free composition plus ^ 
multiple choice intcrprctatidh ^ 


.•700 


Paper 1 '1-6 
i Paper IV i 

-1- 


Papers 111 and iV - | 
specified writing tasks plus 
multiple choice interpretation | 


•683 


I Paper III - 1-2 
1 Paper IV j 

-i 


PapcrsJ A. and i and II - i : ' 
freecomposilionahdJilerature i 
plus trad^iona! interpret..tion 


•737 


1 Paper lA j- B = 1-5 
Paper 11 ^. 1 


Piipcrs 1 A and B and IV_ - _ 
free composition and literature 
plus multiple choice 
interpretation j 


•737 


Paper lA f B = 1-8 
Paper IV - 1 

i 



Note: Correlations iire for 1529 pupils: Cbrrelatjohs for combined papers have been calculated 
by the formula for multiple eorrelatibhs involving three variables. 
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Reference testis u to scale inteirial assessment, it is obviously economica! 
to reduce the test and the marking to the minimum necessary. Multiple-choice 
tests are attractive for this reason, provided that they are validly testing signifi- 
eant English skills. Consideration of correlations involving the maj-ks for Cri- 
terion Tost Paper IV are, therefore, of speciaLmterest, as they give some indi- 
cation of the^ re^ between skills measured by that test and those 

measured by others. 

First, how does the multiple choice test compare witlvthe traditibrial inter- 
pfetatibri test as a suitable reference te^t for Folio Assessment? Their respective 
correlations with Folio marks (for 1529 pupils) were -58 arid ^66 whicK_are 
^o""^ be significantly different when submitted to the 't-test' for the signifi- 
cance of the difference between two correlations with a cfiteribri when the 
variables are also intercbrrelated; he.^ interpretation paper in 

the: Criterion Te&t- peasurcd a larger proportion ^han did the multiple-choice 
test of whatever the^Folib Assessmerit measured. When the same two tests were 
ebrrejated with P-grade marks, the traditional interpretation produced a co- 
efficient of '12 and/^the multiple-choice test brie of -63 - again a significant 
differerice iri favour of the traditional test.' 

Ft will be se^ from Table 5:7, however, that when either iriteipreiatjbn test 
is combinecir;with literature test, and appropriate 

weightinj^ adjustments are made, therorrelations between the cbmbiried writing 
A"^^fP^^^2l^>o'^^ are similar. Part of the 

explanation Tor this apparent contradiction is that there is a clbsef relationship 
between the writing task of,, e.g.. Paper lA arid the traditional interpretation 
test tr=i-56j than between writing (lA) and the multiple-choice_test (r=-45). 
The traditional interpretation test had a greater bverlapj w^ writing test, 
sb that cbmbiaii^ with it raises the correlation with the Folio less 

than does combining writing with the multiple-chbice test. 

In sum, it would seem limt n tradirional /> ypretat foH Jest wouM he a heiter 
yeferenci^ test iiim^^ one, if only the, iii terpretatioTj test were used: 

If, Jioweven writing or writing xmd iiterature are combmed mth the Tnterprytatloh 
rest - which would he desjrdhte - a wide enough range of English skUis is covered 
whether the interpretation test is of the tradiiionai type or in muitipie-choice form, 
provided that suitahte attention is given to a^ustjng ihe_ relative weightings of 
writing and interpref atioh according to the nature of the tasks set. 

A reference test need not necessarily be applied to allj^upils. A system iri 
which a limited selectio^^ tasks froma suitable proportion of pupils' 

folios was actually marked externally by SCEEB could establish pass rates and 
the appf dpriate dispersibri each school. The sample of pupils from 

each school would have to be a suhstantiai om to ensure adequate represeritatiori 
of all levels of achievement. 

Moderation of standards arid spread of marks a reference test would 
prcrt>ab!y be sjaU would not solve the problem posed by schbbis 

(such as schools IS^nd 35 in the project) iri which the jteachersjjroduced a rank 
order riiarkedlyjdif!e_rent:fr^ which other assessors would draw up. If the . 
SCRE Criterion Test rank order is taken as the criteribri,_thejeachers' rank 
brder iri rib schpb[ iri tKej)lr^^^ be said to be better than the O-grade 

one, while in schools 13 and 3i the O-grade ranking was clearly superiprvThis 
fact is apparently iri conflict Mthit_he_w_idejy held view that teachers rank pupils 
best. They may do so within classes, but^iot^necessarily on a whble-schbbi basis. 

'It shQuld be^noted that the traditibrial test required the pupils to write a summao': a fair 
evaluation of the multiplc-chdice questions Jvould compare them with the iraditional test 
without the sumniary: , • 
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is likely that the bes^^^ way of reducing the effects^n 
of conformity of judgment about pupils' work is to combine internal assessment 
with external, 50 that some account is taken of both the teachers'_arid the 
examiners^ judgiTT^tSi: counterbalance Is applied to any extreme diver- 

gence on the part of the former from the conisensus as to quality in English. 

:Systern of com^^ ajid:extcnial,^is3essmerit (the advantages of 

which are set out more fuJiy in Chapter VII) would allow the possibility of a 
yariatibii of the mon above:: work: for external mark- 

ing could be extracted from «ve/>LpupilVfiDiio, so that the extracts became, in 
effect, the extenlal examinatibn^The SG^^ extracts could be 

the reference mark to serve as a moderating touchstone for the rest of the pupils' 
work,^and it cduld also be combined with the imernal mark. The logistics and 
empirical effects of such a scheme would, however, require extensive investi- 
gation prior to implementation. 
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CHAPTER VI 



MODERATION: SUMMARY OF FINDINGS 



1 . The- Visiting Moderator scheme faiied to identify one school which proved 
^^_be exceptidnall^ and Awp others jni whieh 

the correlation between Fpiio marks and Criterion Test marks was shown to 
be sigriificmitly poorer than that for O-gfade marks with the Criteriori Test. 
[t also failed in all eleven schools to adjust the numbers obtaining awards at 
each band to match the O-grade pass fates. This latter inadequacy could, how- 
ever, beiexcusedt^ since there were clear iridicatibris that the numbers falling into 
each band in the O-grade English examination are determined to a significant 
^^^gree^nbt by the the standards achieved, but 

by the statistical standardising propedure, which spreads out the usually very 
bunched distfibutibn of O-grade marks. 

\. ''^^('C^^'M ^ f^^^^ f^JM^ IS thai the 

SCRE project has not e^tabiished that such a scheme wouid guarantee compara- 

af tolerance, which are severity or leniency not exceeding one S€£ range per 
candidate. If comparability of marking standards and of pass rates at each band 
is the priority, u common reference test is needed, 

2. The O-grade examination provided such a: reference test for the project. 
^f^U^S 1^9^^ Marks did effectively correct overall severity 
or leniency in the schools and xxffected the schools' dispersions of marks so that 
pass rates at each band In Folio were coir^drable with those in O^rade, 

3. The effect of scaling on internal awards to individual pupilsi depends 
partly on the degree of coitelatidn between Folio arid O-grade. W^e/i /^^ 
CO rr elation was -70 or better there remained siill t:p to 40% of a school's can- 
didgi£s who received inrernal marks Qafter scaling) differing by the width of one 
flil^dnd p^^^^^^^ O'grade marks. Combining internal and external ' 
assessment may be the best way (^dealing with disagreement on ranking. © 



^- U was fdurid, feiher, tha high cdrrelatidri 

between Folio and O-grade scaling against O-grade could lead to inaccurate 
^^^^^s for individual pupils unless t^ quality of the school assessrrierit had 
been consistent in all classes. In-service training, including Trial Marking exer- 
cises, would be hecessary to ensurje that teachers were aware of tjie med for 
assessments to be siandardised within each schooL Adherence by the teachers to 
instructions to producer schpoi rank order rather than to assess each class^^a^ 
ratel^ would be_^^^ seen in the following chapter lhat a common 

interpretation test is probably jilso requisite, > 

5. Central to the problem is the SCEEB^tandardising procedure, which, iti 
the case of O-grade E sigiifieant difference to the numbers of 

^pupils awarded high and low bands. The typical bunching of O-grade EnglisA 
'^W^^ P^^^ l^^ 'e^PMl^^^lo^^ In Internal assessment, mayM^ue to markers, 
and teachers' unwillingness to award high and low marks - the researchers certainly 
did note a tendency to unrealistlcldealism when teachers wre ask e^^ in tke pri^ 
UmMrjf stages af the project what they expected A pass pupils to be able tot 
achieve - or ft may reflect the true distribut ion, of English skills In the sixte^n- 
yedr-old population as far as it is possible to distinguish one pupil's commtmd of 
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fJ^M^If'om others'. Jf jJie is the case, disfjiiciiom bei\s:eev^ pijpih obmrmng 

contiguous ranges and even between those obtaMng hands B, C pnd I>in either 
FoUo or O-grade are much the standardEed ranges m^d bands 

.suggest, and markers and teachers should be taught to think of discnmin^^^^^ 
nor in terms of ranges. Or, with even stronger reason, of continuous marks out of 
30. 50 or TOO, but in terms ofbamh. 

J^. it yntght have been better to scale folio ?narks against rdw O-grade marks. 
This would havie effected any necessary correctTon of overall severity or leniency 
JL^^PkT^^ dispersions of marks, but by making smaller 

adjustments than were mad^e using the method employed in the prefect. Disagree' 
'VW^A on ranking wpiM have been so grossly exaggerated by the stretching 

out of three-mark categories to five-mark ones. 

1. ^ The results of comparisons between the 3CRE Criterion Test elements and 
1^^^^^ "^^^^^^^^ suggest that an external 

reference test should contain at least writing wid interpretation elements. 

3' If the test is a multiple-choice me, or iiterature element is 

desirable as well as composition, but is not essential. The combination af all three 
of task, if appr^opria^^^^^ weighted, covers a range of skills comparable to 
that^/fwasured by Folio Assessment in the profect; the combination multiple- 
Mf'^f^^^^^^ alone, though not so satisfactory, also 

correlated adequately with Folio Assessment. 
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iE-HAPtER VII 



TWO FACTORS AFFECTING THE QUALITY OF ASSESSMENTS 
L Inter^Marker Inconsistency 

T^e fcnaw[e<J^e tMi the marking of English compositions suffers from inter- 
marker tncojisistency is now relatively ancient. The findings of Hartdg and 
^^P'^^^ j^ TT^KM^^f'ks I?/* (1936) arid of the International Institute 

Examinations Enquiry in The Mzirkmg 'of Engiisli Essays (Hartog et at, 1941) 
^A^^ I" ^^ct had beneficial irifiuer^^ which 
are still effective. It was the latter report which encouraged a concern for judg- 
ments in the firist instance about the jenje of the writing, this^^^ assessediin 
tennsiGf the writer's success in attaining his object: it also advised against the 
practice of arriving at a final mark for a cdmpdsitibh by sunirriing marks for 
separate elements _s^^^^^ spelling, vocabulary, structure . . . , a warning whic>^ 
is still repeated in the SCEEB instructions to markers. Despite advising means of 
"n&roy|nj5 tlieipsessment of wHtiiigi t^^ authors of the Marking 

of English £sya>'y were so aware of the unreliability of examination essays as a 
"^^^"s of testing the ability to writejhat they recpm trial period during 

which the writing task would be removed altogether from the School Certificate 
Examlnatidn (with the safegua^^^ 

Education might have powers to reduce grants if compositioawerenot taught!): 
'^^.^y^*^^^ PL^sjsimist[c, ha^^^^ evidence 
thajexaminers all meari t even approximately the same thing by the term literacy', 
having noted great divergencies between the marking df the same scripts by 
different examinersiarid^^ exercise, having found that the exami- 

ners disagreed not only with one another but also with theif dwn previous judg- 
"l^Pls. French to reinforce their own findings that different 

experienced examiners are .<iuite capable of awarding the same candidates 
marks which differ by mdre than 25% of the possible total. 

That such problems were still current was suggested by the arguments for 
multiple-marking df cdmpdsitidns made out in MuttljjU Marking of English 
Composif ions (Briiicn, Martin and Rosen, Schools Council, 1966). 

The SCRE researchers, therefore, apprdached the question of inter-marker 
unreliability relatirig to the assessments under investigation not in order to 
establish^ anything new about fflafking in English, but with a view td dbtaining 
some informatK^^ df success with which inodem procedures 

for standardising the judgments ^f markere reduce the effect of unreliability. 
Dr Ballafd, one of the Mthors of the 194J: report wrote in Tlie New Examiner 
(Ballard. 1929): The examiner's blunders are as carefally hidden from the 
P^bHc jazejas the doctor's blunders^ underthejo^^ 

td agree with the implication in that statement of xlellberate cover-up to feel 
that an appraisal of curren t mafkef-iriconsistency in SGE^)^J:rade Engtshjwoujd 
be worthwhile, if it served only td remind all concerned of the continuing 
prevalence of the problems which caused concern td Hartog and his colleagues 
forty years ago. 

A study was accordingly made of variations between assessments df the same 
scnj>t by different m^ As the markers of this test 

were all experienced and satisfactory O-grade markers,^ who were peffdfniing 
^of the project a ta^^^ veiy similaj^to bptfrO-jrad^ Folio marking, 

conclusions may be drawn fr ^m this study, with due caution, about likely 
effects of marker vafiatidn on both schddl arid SCE assessriierits. 
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The usual prbcedure:m a Jtu^^ between marks wouicLbc 

tp arrange for ail the markers involved to mark a common set of scripts. The 
SCRE Criterion Test procedures^ this but did permit an approxi- 

iriatibri to a multiple-marking arrangement: 

_ Those Crteripn Test Papers which were to be double-marked (i.e., I, iH and 
y, the writing and Jiterature assi|?i men ts) from each school were divided into 
eijht_alj>habetieal jroups and, as far as^possible, each First Marker received 
one bundle from each school. The papers were afterwards distributed to the 
Second Markers in su^^^ that one Second Marker received work marked 

by each of the previous markers. TJiiis enabled the researchers to cbmpare the 
marks^of each marker with those of ei^ht other markers who had each marked 
^ portion of his scripts. In the following diiscussion the usual assumption has 
been made that groups of pupils allocated to each marker contained the fuil 
range of O-grade candidates: 

LThe patleni of differehces between the marksiawarded by two markers to 
the same pupils showed up not only consistent differences in severity or leniency 
but:alsp other ways in which marking behaviour varied; differences in dispersion 
of marks and disagreement on ranking, which^niaiibe mlled •inter-marker 
inconsistency VTables ^. J and 6.2 below show marker characteristics for Cri- 
terion Test Paper I (Composition and Literature), Paper HI (Factual and Per- 
vasive WritingJand^PaperjV (Short Story). The columns headed *M^n', 
•Standard Deviation' and •fnter-marker Inconsistency' show measures for each 
r^Ja^^^^^ comparability between markers 

is affected. Differences at the mean may have been caused by consistent sjsyerity 
or leniency. Usually, however, dnej^olu^^^ inter-marker comparability 

to suffer also from the dilferences in the dispersions of marks which are indicated 
by the varying standard deviations. These differences^ of means and of disper- 
sions iCLf maiks are su seep to statistical adjustment: There are, in addition 
to them, differences between markers which are due to genuine difierOTces of 
dpinidn p tq the §ua^ of a script or to factors such as markers' carelessness, 
loss of concentration, or haphazardjiess in applying standards themselvesi. The 
figure Under •Inter-marker Jncbns^ is the standard deviation of the differ- 

ences between each marker's score for each of his batch of papers and the 
score for the same paper given by^ the other marker of the eight who acted as 
second rnarker for that paper. This measure of inter-marker inconsistency is 
affected by the varying dispersions of marks among the markers, but^is not 
directly dependent on them. Its usefulness is in showing that all markers differ 
fromitheir colleagues to approximately the same degree. One marker only, 34 
(^ee Table 6.21 was a ^rpgue', showing considerably greater disagreement with 
the others than any other marker. 
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TABtE 6:1: MARKER CHARACTERISTICS ON CRITERION TEST PAPER I 
(MARKED out OF 60) 



I inter^marker 

AfarJcer -ji^ Mean S.D. In consistettcy 



I 




. 264 


2\'l 


7-85 


610 


2 




258 


32-5 


7-47 


5-01 


3 


(1st Marking) 


266 


32-7 


9-85 


5-81^ 


4 




262 


33-7 


7-30 


5.25 


5 




184 


300 


7-82 


4-89 


6 


(1st Marking) 


275 


30- 1 


6-20 


4-35 


7 




279 


28-4 


8-99 


5-73 


8 




301 


320 


9-62 


612 


3 


(2nd Marking) 


225 


29-4 


7-63 


510 


6 


(2nd Marking) 


182 


27-9 


5-99 


5-13 


. 23 




258 


30-6 


6-76 


5-48 


25 




236 


2S-2 


6-92 


4-97 


26 




265 


.^0-8 


7-47 


5-47 


27 




252 


33-4 


8-60 


5-82 ; 


28 




259 


30-7 


8-73 


.6-50 


29 




253 


320 


8-46 


4-82 


3d 




223 


280 


7-60 


4-59 



Notes: (0 Marker 30 at tended ^ meeting but his aljocation _o^^ 

included some Mst marking' work left over because of the illness of marker 5. 

(2) Though there was no difference in the standard set by 1st and 2nd rriarkers bhjhe 
sample scripts^ at the markcj^' meetings, the overall rrieah for 1st Marking was 
31-3, arid for 2hd Marking 30-3. Means on sarhple scripts were respective'y 30-7 
and311. 

(3) Markers 3 and 6 dld.a doiible^tint, and their characteristics appear lo have changed 
between If^t and 2nd Marking. 

TABLE 6.2: MARKER CHARACW QN CRlTERiON TEST PAPERS 

ni AND V. (MARKED OUT OF 20 EACH) 



PAPER ill ■ \ PAPER V ■ 



Marker 


// 




S.D. 


fnter- marker 
Incotisistency 




S.D. 


1 ; . , 

inter-marker 


Mean 


Mean 


i inconsistency 


17 


262 


' 10-37 


3-23 


2-82 


10-57 


3-56 


! 2-59 


18 


264' 


10-64 


3-08 


2-67 


io-l8 


3-41 


2-71 


19 


260 


9-72 


3-05 


2-46 


9-24 


4-06 


2-93 


20 


. 252 


; 10-53 


2-82 


2-67 


8-95 


3-38 


2-80 


21 


240 


10-66 


2-47 


2-68 


9-32 


3-48 


2-81 


22 


253 


9-83 


3-01 


2-71 


9-76 


3-62 


2-96 


23 


263 


M-24 


2-27 


2-91 


10-88 


3-39 


2-66 


24 


266 


9-78 


3-M 


2-89 


7-04 


3-57 


2-72 


i 


253 


10-07 


3-73 


2-73 


9-68 


3-81 


2-88 


21 


225 


10-40 


2-54 


2-30 


9-13 


3-22 


2-57 


31 


265 


11-00 


3-13 


2-05 


9-01 


3-62 


2-23 


32 


255 


7-58 


2-91 


2-38 


•7-90 


2-77 


2-53 


33 


210 


8-6]S . 


2-95 


2-21 


8-29 


4-18 


2-86 


34 


281 


9-30 


4-39 


3-51 


7-06 


4-85 


4-04 


35 


267 


9-73 


2-94 


2-45 


8-82 


2-95 


2-48 


36 


250 


9-42 , 


3-39 


2-71 


8-52 


3-62 


2-40 



















Notes: 0) Marker j also did First Marking of Paperl. _ : 

(2) Paper lll, overall means: 1st Marking ^ 10-3; 2hd Markirig^^-5. - 

Paper III means at markers' ttlTOting: 1st Marking = 11.6; 2hdlV!arkihg ^ 11.5. 

Paper V, overall means: 1st Marking = 9-5-: 2rid Marking = &-5. 

Paper V, means at markers' mectinp: 1st Marking = 10-5; 2rid Marking = 10-6. 
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Tables 6. 1 and 6.2 indicate disagrcemenls aitioilg riiarkers in statistjcal terms 
only. It is of interest Jo note vvM^^ liiean in rea! terms for papils under 
assessment, and this may be expressed by showing the discrepancies between 
marks awarded to the same script by First and Second Markers. Tables 6 3 and 
6.4 show the figures. 



TABLE 6.3; CR|t ; nJ TESt, PAPER I 

DIH? BNCE BETWEEN FIRST AND SECOND MARKS 
i^EFORE MARKER-STANDARDISATION 

(a) Paper lA - Composition - out of 30 marks; \ 

_ _ _ I 
Maximum diflcrcncc - 12 marks (5 cases) 



No^of marks 
differem 


i % df d\Wbfe 
marks* 


1 

' 'No. of 
pupils 


%(iall_. 
completed papers 


Olo 2i 
3 to 5A 
6 to 
9 lo 12 


\ Icssihail 10% 
i to -19% 
1 20-29% 
j 30% or more 


1142 
' 744 
227 
56 


52-0 % 
34-8% 
106% 
26% 




i 


2139 


100% 


* Note: li was rare for an e^sav io he jgiv^^ than ID marks, so that in effeci only 20 marks 
\vcrc used tor this paper. Percentages given arc, however, based on the possible total 
of 30 marks. 


(b) Paper IB - Literature 


- out of 30 marks. 






Extreme ditTcrences - 13A (1 case), and 13 (3 cases) 


No. (Tf_ marks 
(iitfcrcnt ' 


% ofavattabtc 
marks 


No. <if 
pupils 


% of a ll 
comp feted papers 


bto2i 
.1 to 5\ 
6 to U 
9 to l3 .^. 


lessihan 10% 
»Q- 19% 
20-29.% 

30'!;, or more 


1217 
647 
224 
56 


56-8% 
30-3% • 
10-3% 

2-6% 






i 214! 


roo% 


fc) Paper 1 - A and B - out of 60 marks. 








Maximum difTciehcc - ^ 


25 marks 




No. of ffidrks 
iVifferent 


/n of availabie 
marks 


! No. of 
j pupils 


. %^all 
completed papers 


Oto 53 
6 to lU 
12 to I7.t 
18 10 25" 


less^harh J0% 

40- 49% 
-20-29% 
30% or more 


' 1460 
586 
84 
6 


68:3% 
27:4% 
4-0% 
0-3% 



2136 100% 
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TABLE 6.4; CRltERION TEST. PAPERS ill AND V 



DIFFERENCE BETWEEN J^lRST AND SECOND MARikS 
BEFORE MARKER-STANDARDISATION 

(a) Paper iii - Factual and Pereoasivc Writing - out of 20 marks. 



Extreme differences = 11 J marks (1 case). 9 (4 cases) 



No^:cf_Murks 
different 


1 % Dfamttable 
j marks 


I 

m. of 

1 puptts 


f % of aJt 
j compteted paper.s 


Oto .i 
2 to 3^ 
4 to 5i 
6 to 1 U 


! less than 10% 
10-19% 

20-29_^;__ 
30% or more 


' 952 
t 673 
1 339 
87 


46-5% 
32-8% 
16-5% 
4-2% 






2051 


100% 


(b) Paper V - Response to short story - oiit of 20 marks. 






Maximum difference 


- 1 1 (3 cases) 




No.xrf inarks 
different 


% of a vaUable 
marks 


' No . of 
' pnpits 


% of alt 
compteted papers 


Otoli 
2 to 3.^ 
4to5A 

6to n 


■ less, than 10% 
10-19% 

_..20-29_%_. 
30% or more 


I 963 
642 
305 
130 


47-1% 

'3J-5% ' 
150% 
6-4% 






2040 


100% 



One point about Table 6 J serves to convey the significance of all these figures; 
As u resuir of various combmatToris of severity, tenTency, different spreads of niarks 
gM Jnt^marker incvh^ 31-7% of 2136 pupils, if Kvarked by only one of 

their two jnxirkers\ would hme obtatned on CrTlerton Test Paper 1 marks differThg 

6 - 2S out of ^yhat file other marker would have awarded. Criterion 

Test Paper I was similar in form and marking instructionis to b-grade Paper I 
(though worth 60 rather tM^i^O^^^ pfits extm Literature question), 

The 16 markers constituted as experienced_and satisfactory a group as could 
be found. T^e standard deviatiba marks beinj;:usua_ljy about 

1 2-5 - \h six or seven raw marks are the equivalent of two ranges or one band. 
The implicatioti for tHe rdiability b marking bf Writin^^^ yteratuLre, 
both iri O-grade and in any internal assessment, is clear. // /j quite possible for 
aboiit ^% of ^undtdates^ to obtain an O-grade result which wouTd have differed 
S'- priejull band if their Paper I had been marked by ano ther marker. In the case 
of a further 3% or 4% she discrepancy woutd be two full bands. It is, of course, 
v^f'^ J^f'^^^'J^Pr^ (o fhe O-^rade team that some 70% qf o ver 70,000 

candidates are relidbiy assessed in uisubject notorious for murker'Subjectivity. 

(-T J'^'? ^C*^^ Cr/7mo/? Test study is indeed indicative ^^^^ 
marking behaviour, in absolute terms, some 17 MO ~ 20]000 pnpits probably Jmve 
" ^^W'' ^^'W^^^^^PR h ^['^rA^yr^ _ An^^^^I^ assessed with a significant 

degree af unreliability before niarker'Standardisation. 

: Considerable time ^ a is spent in 'marker-standardisation;, which Ls 

a process by which the Principal Examiner and his cblleagues seek tb check 



that every marker is wofking cbhsistehtly to Ihe standards they set. Their 
method ihM tw^^ First, scrutiny of a small sample of each marker's 

scripts, re&alting in a judgment as to bVefall severity/lenlenc^; a plus or minus 
numerical factor IS sug^^ to correct the tendency if the standafd- 

isers think that there is a consistent trend and that the marker has not treated 
any pafticular ability leve^^ another numerical factor ia provided 

by the computer, when each marker's mean score has been compared wi^h the 
national average mark. A decision isjhen take^ factor to be apphed 

to ihe marker's awards, consideration being given to hiis apparent consistency 
and his^^spread of marks. There is, ho wever, no standardisation of the dispersion 
pfinarks. If the application of a numerical factor would benefit some pupils 
and harm others, no standafdisatidn of that marker actually occurs, unless he 
ij sa mcbnsisterit as^ the remarking of all his scripts. In O-gfade 

English, Paper I, it would be unusual for a marker to receive a factor greater 
than ± 3 raw marks. 

There is also a scrutiny of scripts [ - 2 marks below the C/D borderline, to 
try to ensure that rib brie 'fails' as a result of severe marking. 

The efFect of this moderation process on the results tabulated above is an 
^^P^^^^H^ 'sstje. If cbrisisterit sev^^^^^ were a major cause of dis- 

crepancies between two sets of marks forlhe same scripts, the effect bf SGEEB 
marker-standardisation could be exp>ected ta be significant. If varying spreads 
of marks were largely to blame, standardisation of dispeision, as well ^s bf 
mean, would be needed to improve matters-. Nb staridardisation procedure other 
tHi^H re-riiarfctrig by the Princip.al Examiner's team would be effective if the main 
cause of the problem were inter-marker incbrisistericy. 

^heri cb^nlparisbris w^^ and second markings for small 

groups of pupils it was found that the pattern of discrepancies was sejdom 
^"^'^^'^^t adjustnierit bf the mea^ improve agreement, 

and cases in which standardisation of dispersion of marks led to mudj clbser 
agreement were also infrequent. All tbo bfteii the main characteristic of the 
iraphs^ drawn was disagreement on ranking, even when the markers' inconsis- 
tency as measured by the figures giveri in Table 6.1 did rioj mark them out 
specially from It therefore seemed uniikely that the process of 

marker-standardisation would make great difference tb the riurtibers of pupils 
iawarded alarrtiirigly discreparit marks by the two markers of their scripts: 

That this was in fact the case can be seen from Tables and 6.6 which 
show the improvement. due tb niarker-stpdardisatio^^ which may be com- 
pared with Tables 6.3 and 6.4. (Although the pupils involved here are a subset 
of those in the earlier tables, the figures are still cbrtiparable.) » 

TABLE 6,5: CRITERION TEST PAPER I 

DIFFERENCE BETWEEN FIRST AND SECOND MARKS 
AFTER MARKER-STANDARDISATION 



(N.B. Slightly different population from Table 6.3) 
Paper 1 A - Composition - out of 30 marks. 

Extreme dlFferences ^ 14 (I case), 13 (j case), 12 (2 cases) 



different 


of nxailabic 
marks 


No . of 
pupits 


compteted papers 


0 to 2\ 
3 to sl 
6 to 8i( 
9 to 14 


jess than 10 % 
10-19% 
20-29% 

30% or more 


924 
747 
186 
28 

1885 


■ 49-0% 
39-5% 
10-0% 
1-5% 

100% 



60 



62 



Extreme differences = 12 (1 case), 11 and lli cases) 



^^'^f f^rks X of qvgUable No, of vt^—-- 

di ff^rent marks ptipils completed papers 



Oto 2} less than 10% 993 52-6% 

3 to 55 10-19% 693 36-8% 

^ 6 to ^ 20-29% 164 8-7% 

9 to 12 30% or more 35 1:9 °<; 



1885 100% 



Paper j A and B - but of, 60 marks. 



Extreme dlfierehccs = 21 (j case)^ 11 (i case), 16 and i6J 
(2 cases), i 5 and isl (6 cas^) 











NO'_ of marks 
different 


% o£_avaiJable 
marks 


Noijof 
pupils 


%<>fan 

completed papers 


Oto 5i 
6 to 115 
12 to 175 
18 to 21 


less than id% 
10-19% 
20-2951 

30% or more 


1320 
496 
68 

i 


700^ 
26-3% 
3-6% 
01% 






1885 


100% 



TABLE 6;6: CRITERION TEST, PAPERS 111 AND V 

DIFFERENCE BETWEEN FIRST AND SECOND MARIC^^ FOR 
EACH PUPIt AFTER MARKER-STANDARDISATION 

Paper III : _ :: 

Extreme differences = 8 and 85 (8 cases), 7 and Ik (1^ cases) 



No. of marks 
different 


' % of_available 
■ marks 


No^of 
j pupils 


__ %isfall_ 
completed papers 


Oto !{ 
2 to 3| 
4 to 55 
6 to 85 


less than 10% 
10-19% 
20-29% 
1 30% or more 


\ 705 
1 795 
; 312 

j ^73 


37-4% 
42-4% 
16:6% 
3-9% 












1885 


100% 










Paper V' 




NO ' of mar ks 
dijfferent 


% of available 
marks 


No^d 
pupils 


completed papers 


OtoU 
2 to 3} 
4 to 5f 
6 to 8i 


l^sthan]d% 
10-19% 
20-29% 

30% or more 


691 
767 
328 

99 


40-7% 
17:4% 
5-2% 






1885 


100% 



61 

S3 



the reduction effected by marker-mmtard^^^^^^ in the numher of pupils aWarcTeci 
marks discrepant l>y 10% or nwre (tj'the ayailahle marks by two individuaf markers 
on Paper I A and B is of the order of / % - 2 and on Fapers 111 and V the effect 
"'^•^J''^ /'^^^ (It may be that the 

extremely aberrant performance of one marJcer, 34, had exceptional influence.) 
The average discrepancy bctwccii First aild Second markers j)cr pupil on Paper 
;IA,_C<>mp0_sition, was 3 to 3-5 marks (out of 30) both before and after rnarker- 
standardisation: 

These .findings- havic-important implicatidns for O-grade English markcr- 
standardisatibn^The SC which equalised means and 

dispersions of marks, if appJied to a population as large as the O-grade arie ^ 
about 70 000 pupils wbiild have led to u reduction in blatant uarcliability of 
mark for 1% - 2%, or about 1000 pupils: About 30%, or some 20 000 pupils, 
would still have been. Jeft witli.a.marJc-difFering.b^^^ out of stxty from- 

tJie one another marker would have given: SCEEB marker-standardisation 
probabJy hasxjvcn smaller beneficial effect than this, because it does not affect 
dispersions of rn arises whether the annual hard work on 

marker-standardisation by the Principal Examiner aild his team has a significant 
enough effect to justify it. 

It may be felt that the I^^^^ Js justified if even a 

few pupils thereby achieve a fairer award: It does not seeni likely that mbdifica- 
9^ the procedures w^^ effective marker-standardi- 

sation, since the bulk of discrepancies between markers appear to be iil that 
category of incbnsisteTlcy , ha^hazardness, org^^^^ opinion about 

the: quality of individual scripts which precludes the regulation of markers' 
awards by statistical adjustment. The prbcedure ih current use could cbntiri 
to pick out very obviously aberrant markers - though it had apparently foiled 
to note over several years the. distinct lack of comparability between the judg- 
rncnts of thP: rriarker numbered 34 in the SCRE project and tho.se of mo.st 
other markers: 

Are there alternatives? If the external Dxamination of Writing and Literature . 
^•Isacrbsarict, ther^is a strbrig case for doubte impress marking as described 
by Wood and Quinn (1976) as a meiins of reducing the effects of inter-marker 
inconsistency. Alsb a addition 
of interna! and external assessments, provided that both are measuring some, 
at least, of the same aspec^^^ the advan- 

tages of simultaneously helping to combat marker-inconsistency and taking 
account of different but equally valid responses tb pupils' writmg, by increasing 
the number of judghierits about that writing which contribute to the as.sessment^: 

It can^bc shown th assessments of writing and literature 

does lead to closer a ••cem-^nt bnween marks given by teachers and examiners 
o'lJ'ic o^e hand a^^ -x critcriorj measure pri the other. Figure 6.7 contains a 
.series of graphs in wh ' on;; assessment is plotted against one other (Fig 6.7(a) 
(yjr J'^vb assess together: are JloUe^^ one other (Fig. 

6,7(c) and (d)), and t^v^; si m' lrly combined assessments are plotted against two 
others, also added (Fi^^^^ 

that combining the Foli ap. J '''^-gr: 1 assessments increa.scs their agreement 
with the Criterion Test \i? r -m^M?, of -iting. 

\ Sec Britton, Martin and Rr;.- . v ' chv- -s Cou > - (1966): 



FIGURE 6.7 SCHmL 15: COMBINED ASSEskMENfTS 
a) Composition-and Literature in Folio 
against one Criterioh Test paper 
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c) Foiib anplO^rade I against one Criterion Test paper 
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d) O-grade Paper 1 against t>vd Criterion 



10 



20 30 40 50 

CRITERION TEST I + HI 



60 



70 
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c) Foiio and G-grade I against two Criterion Test papers 



50 



.40 



30 



20 



■ - • • 



I- :• 



10 



10 



20 30 _ _ 5b 

CRiTERibN TEST I + HI 



60 



70 



The effect was also tried of co^nbining Folio and 0-grade tGtalsj[i.e.i Writing, 
Literature and Intisr^retatiOT with the enterion Test totals! 

Does the combination of JFolio and O-grade correlate better than eitheralqne 
W4th_the Criterion Test? Table 6^8r ^hpws the various Peareon correlation co« 
efficients: for each school. Two means of combining the Folio and O-grade 
indiyidnal correlatjons were employed, llie^first^ designated in Table 6.8 as 
'Folip^pkis O-grade against Criterion' is that of averaging the two correlations^ 
T' other^ called 'Folio x O-grade against Criterion' is calculated following a 
f 111 ula which a<S_usts the relative contributions of Foiio and O-grade so that 
the correlation obtained between Criterion Test and the combined Folio and 
O-grade is the best possible from any h'nearly additive combination of the 
latter tvvo^. 



1.(2 + 3) 



^ 0^2>'l^ + g3>'l:3 
>/<y2+2/'2,30-2<T3 + <T3 



f - /''l.2 + '*lv3~2''l.2'*U3''2.3 

1.23 — * 
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TABLE 6.8: COMBINED POUO + 0-GRADE v CRITERION TEST 





















Folio 




1 .. Folio 


FoUp x Qr 


\Fplioplji:LO - 


School 


// 


. ogalnst 


_ ogainst _ 


I gainst 


' :grade_against_ 


\ grade against 






Criterion Text]Crit€rion Test 


j O-grade 


Criterion Test 


\Cr iter ion Test 


n 


143 


•14 * 


•69 


•68 


m 


' ^75 


13 


163 


•68 


•85 


•61 


•87 


•86 


14 


21g 


. •83 


•81 


•74 


•88 


•88 . 


15 


154 


•84 


•84 ' 


•81 


•88 


•88 


21 


93 


•86 


•82 


•83 


•88 • 


•88 


22 


'.94 


'M 


•85 - 


•79 


•89 


•89 


23 


81 


•84 


•87 


•79 


•90 


•90 


24 


70 


•89 


•88 


•84 


•92 


•92 


32 


268- 


•83 


•81 


•74 


•89 


•89 


33 


45 


•55 


•71 


•39 


•77 


•76 


35 


112 


•63 


•84 


.59; 


•85 


•82 













T^^tije 6.8 shbw^ t^est possible cc^^ of Folio larid O-grade 

was barely better than simple averaging of the two. in all cases except school 
35 the average of Fdlid and O-grade correlated slightly better with the Criteribri 
Test than either Folio or O-grade alone^ though the differences were not large. 
While the advantages of combining Fojio and O-grade are not Very 'great as 
compared: with O-grade alone^ if it is desirable to have an internal assessment 
- for educational reasons, it is heartening to note that^ev^n in the two schools 
where the Fo I i o assessmen t cp rrelated sijri ificari tlyjess wel V thaji O-grade again st 
the Criterion Test - schools 13 and 35_- the combined assessments were dis-^ 
t Jiicjtl^ better than Jhe rela tjy ely ' poor j^li o asse ssmen t alon e an d jrib t sign ifi- 
cantly diiTerent from the O-grade as compared with the Criterion Test, it would 
seem that the effect bf Unsatisfactory characteristics ihfl uencirig the Folio marks 
in these two schools was noticeably reduced by the simple averaging of them 
with O-gfade marks. 

Equaiiy weighted tntemai and exfernat assessfiiehts woutd, on ttus evidence, 
probably he satisfactory, _ L < 

A Note on tfiB^jatvarerof the Writing Task and Reliability of Markvrg — 
_ ::A recon siderat ion of Jn ter-nl a rker in con sistency in the Cri t eribn Test wi IJ 
find indications -that thfie was probably jess disagreement on ranking among 
the markers of Paper IIlV iji which specific tasks were set with defined ujpbses 
and audiences, than in the free composition of Paper TA. Table 6.9 compares 
the two sets bf figures. 



^ TABLE 6.9: MARKEJEt-lNCONSISTENCY ON PAPER lA AS 
COMPARED WITH PAPER III 



PAPER lA (marl 


:ed'out of 30) 


PAPE 


IR m (marked out of 20) 


No,Qf.mar/cs 
dtffieteni 

Mar ker A - 
Marker 2 


available 
marks 


Vtipils 


• 

contpleted 
papers 


No^ofmarks 
different 

Marker 1 - 
Marker 2 


%of 
available 
marks. 


pupils 


completed 
papers 


, 0-2* 

3-5i 
6-8i 
9-12 


less than 

10% 
16-19% 
20-29%. 
30% or 
more 


1112 

744 
227 
56 


52% 

34-8% 
i6-6% 
2-6% 


d-ii 

2-3i 

4-5i_ 

6-lli 


less than 

105^ 
10-19% 
20-29% 
30% or 
more 


952 

673 
339- 
87«" 


46'5% 

32:8% 
16:5% 
4-2% 


21^9 


JU)0% 


2051 


100% 
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-^ strict cbmparisohib^^ marking of the two tests would take account 

of the real, as opposed to the nominal, weightings of each^ Without calculating 
real weijrhtings exactly, noted: that: the full scale of 20 marks was used 

by Paper III markers, whereas it was rare for a script to receive less than 10/30 
on Paper I. The two tests actuaU therefore, fpughlj" comparable mark 

scales, and, in absolute terms, five or six marks oji one had approximately the 
same value as five or six marks on the other. A legitimate compLrfsbn can then 
t>e _niade b9tvveeri the percentages of pupils wjth discrepancies between Marker 1 
and Marker 2 of- six or more marks. On Paper 1 the figure was 13-2?^, on 
Paper HI, only 4-2%. 

the arguments against one option composition tests are that they may be 
^^vamageous or disadvi^^ pupils and that they would probably 

have the eftect of limiting the kinds of writing undertaken in schools. In the 

: P^'^sent state of knbwl^ know whether the gains or losses to 

individual pupils due to lack of options would be greater or less than those 

. caused now by markef-incbn top many options: it would 

be ;ui interesting piece of research to find 8uf . The effect on curriculum might 
be Jcsscned if internal assessment of writing and literatiire were jiven significant 

* yvcight ill the overall mark for English and pupils were obliged to produce for 
assessment in school writing of various kinds. The curriculum could then actually 
lead assesiment^as^^^^ Committee (SED, 1977) and many others have 

wished, and teachers could use assessment to establish whether particular skills 
they have sbug^it ta help: pupils develop urc in Tact exhibited. Lip-service js 
frequently paid to this modeiiof assessment but it is probably nbri-existent jn 
Scbllanid. for S4 English. If EriglishjMchefS: real^^ believe in it, it is their 
responsibility to declare: 'These are the kinds of writing and literature we wish. 

-^^ t^?ch: let them be given significant w^^ The external 

examination would then be freed from its present impossible role of trying to 
achieve^gliable discrimination among jpupils while at thejariie time seeking to 
enc^ourage. Of at least allbw^ in schools an infiniteiy wide variety of courses, 
texts and emphases in English teaching. 

Sb long as significant weight were: give^ to internal assessment of several 
types of writing, it would be possible to set a single clearly defined task, without 
choice, in the cxtemal paper to facilto^^ reliability of marking. Variation 

from year to year of the nature' of tlje external test would avoid the danger of 
schools devoting inordinate, amounts of lime to preparing^pupils for one kind 
of writing only. If. however,, the external test is also to b«: used to moderate 
the standard and dispersion ^)f marks of the intern'^l assessment, care would be 
necessary tq^hMre that it did test a fair proportion of the writing skills also 
tested by the teachers. Table-5:7 (page 50> ishows that tasksjikc those in 
Criteribn Test Papers T a^^ traditional interpretation test 

form a .reasonably satisfactory refevence test for Folio Assessment: further 
research WOTld be rcquire^^ t^> .establish the juitability^ or unsuitabiiity of any 
other kinds of specific.'task, ..uch as speech writing, before they were employed. 

IL The Influence of the Task aiid Choice of Tasks 

- - _ — . * - 

Evidence that a wide choice of ouestions or assignments contributes to Unrelia- 
bility in examination marks geh|i^lly has been gathered by several researchers^ 
most recently by Willmott and Hi3^(l975) in "Q' tevel Examined: the Effect 
o£ Question Choice'. There is, however, sbme difi'ererice of opinion as to the 
' 'Tects of guestion choice in English examinations. Willmott and Hall refer to 
J: M: Stainaker's essay on The essay type of examination' in Liridquist (1951), 
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Educationui Meusurumentr in which he argues that allowing a choice of essays 
^iflplM th^^assUi^^ would equally weil facilitate the 

measurement of some general ability to write^ If thiis dubious assumption is 
cbrrect, no dptidna^^^ not sd, and the successful 

pupil can in fact write well only on the one topic he selects, *the significance df 
the pdjsessidn df this ability is so .iTicult to integ^ret that one may question 
the use of the results'. (Wjllmott and Hall, 1975, page 8:) On the other hand; 
Wiseman ^d Wrigley (1958) found that the real differerices-whicH occjirred 
between the means of children selecting diflferejit essay topics in an English 
examination weie Ujrgely due to differences df ability in the children^ aridithat 
topic^hoice jri En^lLshrCo^ unlikely to introduce substantial error 

into marking: Wiseman and Wrigiey point out that abdlitidn of chdice might 
might riot affect reHabili^ty ad^^ More recently, Adams and Pearce (1975) 
quote H. Rosen's PhD thesis for London tfiiversity as. having established that 
a pupirs grade is determmed as rnueh by his^chaic^^ topic as 

by any other factor. Stevens (1970) shows that grades awarded to some 'A' level 
English -Literature candidates depended to a si^ificant degrecj pri their choice 
of questions. Research does not appear to have setried the matter: 

Bissu^ested in the other section of this chapter that the vagaries of jnarJcers 
might be more easily kept in check if all pupils tackled the same specified writing 
task^ or, preferably, JeyeraJ such: ^ One of the advantages of Foiio Assess- 
, , nicnt is that it can test performancetin several kinds of writing arid so reduce 
"whatever problem are caused by allowing choice of style and topic in a single 
examination. 

At the same time as Folio Assessment reduces choice-pf-writing-tdpic prdb- 
l^rils, it unfortunately seems to intrbduce a factor <ietrimeiUal to comparability 
^ of assessment which is not present in an external examination: av variety of 
inteipretatidn tasks ^for diffbrent pupils under the sariie assessriierit. 

The discussion of bunching and marker eifects in section I df this chapter 
has cOTcent rated on ^^^W^^ and Literature. It is appropriate lo give some 
attention to the credentials of Interpretation as a testing tddl before cdmmeritirig 
pri difticuiye& oTits use Folio Assessmerit. (The term 'interpretation' is here 
used as synonymous with 'comprehension':) 

(b) Interpretation Testing 

Given passages df suitable diffic^^ questions^ an Inter- 

pretation test ought to discriminate more finely and more reliably than com- 
position: marks are acqiured, not on the basis df a hdlistic iri1press[dri, biit stj^ 
by steplas individual ques^ are correct, and it is possible to limit the eff'ect 
of the markers' subjectivity by a predetermined marking scheme dr one agceed 
ateori^ the riiarkers. There is undoubtedly some marker inconsistency in the 
marking of G-grade Paper If and similar_tests set in schooI^,_but probably 
corisiderably less thari jri the riiarkirig of Com^c^jtipn^ The re^^ could 
not carry out for Criterion Test Paper f I a cross-marking study like that-made 
for the dduble-marked Papers, but ca^^^ of both 

st£mdard ^nd dispersion of marks among the eight very experienced markers 
of this test. Examples of diff'erent distributions df marks aregiveri in Appendix 9, 
and the means arid standard deviations for all eight markers can be seen in 
Table 6:10: 
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TABLE 6.10: MARKER CHARACTERISTICS ON CRITERION 
TEST PAPER II 

















5 Marker 


Mean \ 


S.D. 


A^o: of scripts 


.9 


20' \ \ 


70 


283 


10 


18-4 i 


6-8 


273 


n 


1 8-3 ; 


6-9 


347 


12 


18-4 


66 


290- 


13 


200 i 


7-3 1 


287 


j4 


i9-5 


7-3 


291 


j5 


2j-7 : 


>3 


mi 


16 


202 


8-i 


260 


Overali j 


19-3 i 


7-3 1 


2i33 



Note; The figures for markers 15 and 16 are after the appl!cat!bn "of a- 
satibh factor. 



1 marker-standardi- 



A featar^ of the consideration of the martcmg of Criterion Test Paper II was 
the fact that some markers produced an overall mean comparable to that of 
their coljeagues by^distinct severity In marking the summary and distinct leniency 
on the rest of the Paper^ viee-yersa. : :MuU of 
O-grade or Criterioii Test Paper II scripts would be a valuable exercise which 
wpiiW hdp in tfe rational judgments about the possible use of 

multiple- choice tests at O-grade. 

Whatever the reliability of interpretation tests, the»r main disadvantage is that 
^^_^y_^re not e£4sy to construct validly. Problems arise in choosing passages of 
appropriate style, content and difficulty for the standard of the examination 
and tor the ratsffc of pupils taking it. It can also be argued that Interpfetatjon 
tesjs, as traditionally set, are unrealistic reading tasks which are hot valid tests 
cf pup}jsV_i,iieraI reading ability, because nobody ever reacb anything in the 
V the tests r^quire,^ except in examiriations. WriHng unamSigUCUo y 
lis ■ a; the right Je\?el of difficuhy is anothei" tricky Job. The case that^schools 

u ujd asSiQss reading i n _spme^ way pt her than Jhe t ra j Ltjonar intei^retatj on test 
ci-« id certainly be put. l^e difficult:csx>f ensuring comparability of interpretation 
* k ' ' ' ffcren t sch c ol s arid o i fferen t usses had been expec ted ; the re- 
searchers beheve that they have in fact foRRd some statistical evidence, discussed 
below, which shows the detrirhciital effects for accurate ranking of njjrt- 
comparabiiity of interpretation tnsks used as the bdsis lor a comihori assessment. 
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FIGURE 6.12b SCHOOL 24:^ ^ - ^ , 

O-grade Composition and Literature against 
Criterion Test Cdmpbsitidn arid Literature 
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FIGURE 6.12c SCHOOL 24: 
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z There were 'class effects' in a number of the 'whoie-schoor assessment sr^inals^ 
po&sibJy due to a tendency to |:ive higheF marks to all pupils in top classes 
andilGw ones to all pupils in bottom classes, or to failie to adhere to the 
instruction to mark on a team basis. Another causeicaiM also, be identified. 
In order fo save teachers; time, the instructions for marking Folios did suggest 
that the marker, assessing the folio work of puMi he hadzribt taught, could 
make a speedy judgJneiU the standard of interpretation marking previously 
carried out by the class teacher on work in the folio. If he was satisfied that it 
was^m fact of O-grade stan^ accepfts^ valid any interpretation 

marks given by the class teacher, withput himsflf having to re-marJe^veiy 
question. In effect, then, the tcachej-s were required to mark only writing and 
literature _^s a team: they could, and probably did. accept the class teachers' 
interpretation^arks. Any other arrangement would iiLfact have been imprac- 
' ticak -Qass ie#2c^^^^^ may then have been caused by varying standards of inter- 
pretation tasks and marking between classes. The clearest examplc of the effects 
different inteiSretatibn w^ classes is provided by school 24. 

Figure 6.11 shows that class 2 in that school was more leniently assessed on 
Folio than other closes. Figuresi6. (KaJ to (c) prove that the Jeniency was 
mainly in the assessment of Folio fnterpretatloh. Figure 6.12(ai plots Fb ho 
Composition and Literature marks agaihstithosczfor the Writijig and Literature 
elenieiits in: the Criterion Test. Only a Jew pupils in class 2 overlapping oil 
Criterion Test marks with some of class 3 were giveiv hijher Folio Composition 
and y terature scores than the class 3 pupils. The same applies to the assessment 
of writing and literature in the O-grade (see Figure 6.12(b)). Figure 6.12(c) 
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however, shows that on Criterion Test Papier IritergFetatibh, classes 2 and 3 
were^^riot cIeMlyJistiBguish^^^ from one another^ whereas ^lass 2 obtained con- 
sistently higher marks for Folio Tnterpretatidn than class 3. 

The combined difficuUies of Jack of time to stmdardise team m^^^ 
PP^^^^^ W^f'P/'^Mf'o^^^^^^^^ classes and of setting passages tmd ques- 

tions of comparable difficulty in different classes und d^erent schools lead Yo the 
'^M^^^'^'^J'^^^^ 'f J^y^M^^ld^^^ to be retained as a means of 

assessing reading ability on a national basts or on a schooi basis, it Would be 
adyrsablefor ali pupils to fake common tests, nationally, or at least within each 
school. 

iThe concliijion as tentative because the researchers are not convinced that a 
single national interpretation test is, as is cdnimbnl^/ behevediiai valid and 
reliable test of reading ge^ It'is as hard for the setter of O-grade Paper FI 
to produce a thoroughly valid test as it is for the teachers, given the dLfficuIties 
listed earlier in this sect^on -irid the O-grade has to cafer for 70,000 candidates. 
IVIarker unreliability affecting Paper II is as yet unexpldfed. 

It has been suggested in the preceding chapter (page 51) that the cdmmbil 
external interpretatidn test ^s^^^^^ be a: multiple-choice test if it 

is the p/j/j reference test employed. If, however, it is combined with writing 
and literature tests^the interpretation test, on the evidence orM»e SCRE Criterion 
T<«t Fesults,; rnay equally well be traditional or multiple-choice: in either case 
the cornbination of writing, literature and intcgy;etatidri lasks cbvers a range 
bfirea<Jing and-^^^ wide enough to form a satisfactory reference test 

for Folia Assessment, provided that apprdpriate attention is paid to the relative 
weightings of each element. 

The S€ RE data could probably ^^^^^^^^ more information than has so far been 
obtained about the quality of current methods of testing mterpretatioh. In the 
'"^^/''^^ ? •^M^'^ preferable (as a means 

of discriminatory testing) to one allowing each teadier to find or invent his own. 
^^J^^ ^ ^y^^'^^^ ^^W^^^ assessing redding per- 

formance, by means cf traditionai interpretation work or by otiwr means such as 
tJie tasks set Jn Criterion Test Paper V / and combinations of jn^^^ might he 
dl^rgpri^l^!.^'^^^^ examination mark might be combined with jin hi" 

ternal impressiau mark for various' kinds of mterpretation work. 
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CHAPTER VIII 
SUMMARY OF STATISTICAL EVALUATION 

1. Seven ourof eight se^^ In ^whole-school assessment' produced 

a rank pfder of pupils jiot significantly different from that of the O-grade 
examination in assessing achievements also measured by the SCRE Criterion 
lest; m class assessnient' schools, one out of three did so. 

^' assessment is desirable, rather than class; assessSent: A com- 

^.c^^r^"^^^'^ 5^ T^'¥^ the school is desirable, as well as awar^- 
_ ness of standards elsewhere acquired through, e.g.. Trial Marking Exercises: 

3. Organisational and management difficulties^ especially lack of time for 
llf 1^''^^^^!^,^^"^^^^^ affected the quamy 'f LJ^IsSs^ent 
r^^^'^-^'™^^^^?^'^"^^ alsb^revented three of the experimental 
T ^o-RPf r^^^ng the project as agreed, and caUSed a fourth to 
provide only partial data. / luuim lu 

IH Jhe^ssessment of writing andrespdnses to literature the following fictbrs 
probably contributed to unreliability in both FoHo and O-grade. 
(a) Intei-mai^r Jnc^nsistency, causing some 25% of pUpils to obtain 
marks a full bmd or more different from those they would have re- 
ceived from a liferent m Marker-standardisation had little in- 
nuence on this mconsistency. 

(bj Lack of perceived distinguishing qualH in pupils' scripts, so that a 
very bunc!ied_distributibn of marks resulted: (O-grade results aspublicly 
reported do, however, give an illusion of clearer discrimination between 
pupils bec^use^of the SCEEB scaling procedure:) English teachers and 
markers should leam to think of discrimiriatibni especially among the 
arge -mass of average; p_ui>ilSi in terms of the five or six bands rlther 
than in terms of continuous marks out of 50 or 100. 
(c) Variety of choice of tasks/stimuli. 

In the assessment of interpretation (or cdmprehehsibn), the rollowing points 
emerged from the researchers' experience. . 

(a) There il a need Tor further study of the validity and reliability of this 
traditional method r*" testing reading ability. 

(b) In the meanUme Jf com^^ desired, common interpretation 
tests^are preferable to a variety of tasks set by different teachers in 
diiterent classes or schools. 

6. Schools' pass rates. at each band diribred from O-grade pass rates after 
.standardisiatibri of O-grade marks. The schools awarded too many Cs and 
too few As and Fs. This mismatch with O-grade standards was correctabie 

^raj.^by the appl^ Folio marks of the mean and 

standard deviation of its marks in the examination. 

7. Thls:j_c_aling process significantly increased the match between JFblio and 
Upgrade awards m particular schools only in the circumstances where the 
school s ranking had been comparable with that of the O-grade examination 
Of all abihiy hv^is. but there were discrepancies between Folio mean and 
O-grade mean and to Folio dispersion of marks and O-grade dis- 
persion^Disagreements on ranking were exaggerated by the scaling process. 

^Alternative approaches should be investigated to see if the exaggeration 
ot ranking disagreements could be reduced while mean and dispersibh are 
; appropriately standardised. 
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8. An external reference test against which to-scale internal assessments h'ke 
the Folio Assessment in this project should cqritliiri at JeMt w^ 
interpretation elements. If appropriately weighted the interpretation element 
could consist of a multiple-choice test. ^ 

9: Provided that they areall validly assessing some conimon aspectsdf 'English', 
combinations of _as^^^ of Writing and Literature, can im- 

prove discrimination and reliability. 
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CHAPTER tX 



PRACTICAL PROBLEMS 



I. Problems in bbtaihihg Accurate Data 

Failures by the schools, for whatever reasons, to provide ^11 the_ required data^ 
or to send it in the^tes^^^ may indicate the kinds of communication 

prpblems and recording errors likely to arise in a real internal assessment system. 
They are impbitant also^^^^^ bear on the degree of correlation 

among the three assessments made in each school. 



X^) it^MJity to provid^^ 

The.effect of pressure of work on teachers, whjch::is discussed in the foJIowing 
section, ledjq the failure of two schools to provide folio assessments at all, to 
> P/*2^^^i"fi assessments for a proportion of pupils only, and to a fourth 
school being unable to administer the Criterion tfest. 

(6) piaccuracy - ^ 

Some inaccurate reporting of scores and miisinterpretatibri of instructions oc- 
curred in the other schools. 

^ l^^^opjs had beeii. asked to report three marks (composition; literature 
aiidi interpretation) and assign a *range\ (1-14) based on the total of these, 
following the pattern of the scaled O-grade marks, i.e.: 

range 1-90-100 2-85-89 3 = 80-84 4=75-79 5=70-74 
6-65-69 7 = 60-64 8 = 55-59 9 = 50-54 10=45-49 
11=40-44 12=35-39 13 = 30-34 14=0-29: 

Four variables were therefore required (three marks, one range). Some 
teachers sent 5 variables: 3 marks, a total and a raiige. 

Among some 2000 pupils, 227 diiscrepancies occurred between the total mark 
(whether recorded! bi the t^^^^ or not) and the range assigned (e g:, a total 
of 54 being translated as:Range-8 instead of 9): Only seven of these proved to 
be copying errors by SCR E staff. 

So about ! 1 % of the folio assessments from 10 schools were in some way 
Unsatisfactorily reported to SCRE. 

Sorrie H3tlier: :e^^ arose because some classes in two of the Strathclyde 
schools Jiad recorded the three components of the folio as ranges jratherihan 
•ll^rks. This pr^^^^ because of a changed instruction to the schools: 

they were asked to record their Autumn assessments as fdurteeri ranges, but 
[ater asses^^ three percentage marks and one (total) range were 
demanded, because it had become clear that the statistics would require a wider 
range of marks than 1-14. 

J^''^ Z'''*^^ }^oUM have b^^^ to ask for rmges, 

M^l simply fir the three compDnent marks xmd the total out of 100, It would 
then have been desirab!* d check the totals immediately and ask the schools 
to clarify those marks in \>hich there appeared to be an error. 

( ^) P ^ U^ ^/jR t^^ 

When all these various reasons>had been taken into account, substantial dis- 
crepandes between the tgt^^^^ ar^signed range still remained 

for a considerable number of pupils in particular schools or classes. Unless some " 
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teachers were excessive , '^arck jliese wen ;ji^»>; abiy deliberat: and. indeed, 
in some cases were expi" . "^ indicated as Srwyiu 

- The cases of knov< deliberate discn iancy between Folio " otai and Folio 
^ MSge <)CH:urred in schools 31 and 35 ina *rc i,pcdajiy interesi ng. They suggest 
that some t eachers regarded their .-uf; is' 'abi'ir m English as somethmg other 
t^an whati the work in their loWcv-. '- stilV- Jn some ^ases it wjas explicit 
that the teacher was making a prcviu ; of cifdfmance in the O-^ade in 
assigninj rpges. The ten^^^^ to do i:-. . stronger if pupils had failed 
to complete the minimum number of assignrneiits for the folio asses^nientj 
hence some 'estiiT^^^^ in discussion and written 

documents by teachers at school 35 was the feeHi g they had that some of their 
pupils had undeNperfonnrxed^ certain: foli. v r-iignments, namel/ the 'close 
reading' exercises recommended as a possible 'duTerent^ sort of O-grade work: 
the teachers thought that Unfamiliarity with this kind of work had caused some 
pupils to score marks lower than then 'ability' warranted. 

1^ Schools 31 and 35 may have been the only ones where sucH subjective judg- 
ments were made, or they may have beeil more horiestjhari the others. There 
some indicat/bn th instruction to produce folio marks on a sviiooi basis 
was not strictly implemented in all the schools which received it. 

It yv'ouM see?7j that in zmy >eiiV tntenrai assessment scWme there should he a . 
^^''y ^^^or msfructlon qs to Mat e^^^^ teachers are being asked to assess, this 
preferably being pupils' actual achievements in school rather than putative per- 
f^^^Mf^^^^ 9^ ^M'^f'^^'l c^^ The cpntinuing presence arid influence of 

an external examination may well, however, make such an instruction difficult 
for teachers to adhere to. 



II. Problems for Teachers 

l^) _ Ordinary hMsa^^^ ' ^ 

The project must have been influenced by those ordinaiy annoyances of da 
school life,^^^^^^ a class, which drain 

^way time add energy in disciplinary, organisational, and administrative matters. 

of the j)r^^ teachers and pupils for 

internal assessment was dampened by unfortunate timetabling (e.g. all S4 
English classes last period in the afternoon, except ohe_, which last period 
in the morning!), unavallabiJity of suitable folders, or vandalism of folders kept 
in insecure cupboards in S4's distant annex location. A few teachers seemed 
flisheartened absented and lack of motivation in the Mess able' classes 
the3^ were teaching. and felt that the extra administrative and organ isatibria! 
work imposed by the project was merely that, and did not benefit the unwilling 
pupils. : 

(b) Teachers* attitudes • : . 

^ s^Pflci iiiflu^^^^ beehjhe dejree of willingness Avith which individual 

teachers became involved in the project in the firet place." Some welcomed the 
idea of internal assesSnlOTt a^^^ wbrkeid hard to prolduce iTLatenal for the project. 
Others, drawn in with their colleagues or their Principal Teacher, were satisfied 
with the existing system and, envfsaging nothing more than extra workTesuIting 
from the project, were, in a few cases, hostile to it from the start or, in others, 
indifferent' to its success. A negative attitude on the part of the teachers was 
ilbtLw|de^P^ad^ibU^ was: certainly to be found occasionally, and lack of 
interest, if not hostility, surely contributed to^'the failure of one whole si^.^oibl 
to prbvide the promised internal assessments. 
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A number, of difBcuities arose because of the particular ways iii which the 
Object was cdn^ frorn imperfect cominunicatioh. Some were caused 

y the double intention of the researchers on the one hand to investigate the 
comparability of school aM^ the Criterion Test, 

and, on the other, to encourage more detailed course planning, perhaps i'uiv^olving 
some break with O-gfade tradition, as the esseritjal first step in dc vio^^iiig an 
intemar assessment system. The late arrival in the schools of ti e 'Guidance 
Booklet',^ with its advice on both the stpdardiiatibn of assessm^^ 
design of them, meant that some teachers felt that they had not had tune to 
consider properly the requirements of the project. Some misundersiari(i?n^arpsei 
too»_<>ver the place in the project of the 'close reading' which was suggested as 
a type of work not normally covered_in O-grade crashes and of which some 
exemplars w^^^ The intention of the researchers had 

been that marks for such work, if carried out, would be recofded under Titefa- 
perhaps, /In teipretation'^ but ieveral teachers in one school complained 
that they had devoted considerable time to preparing suitable materia! only to 
find that the Folio AssessiTlent did ilbt lake 'close reading' into account. 

: :iSQme other faijufes in communication relatiiig to the recording <of folio 
assessments are referred to in the preceding section bri prbblerris in obtaining 
accurate data.) ^ 

It: is the view of the research team that confusion was created ih_ some 
fMclfers-Mnds,:nqt so much by the fact that the project was trying to standard- 
ise assessments and encourage course-planning at the same time, as by the fact 
J t_ rolled tb co^^^^ course-planning is part of 

assessment design and that it should precede choice of assessment technique.' 
Despite the double intention, the e^^ the need to 

standardise judgments about pupils* performanca rather than on design of an 
asse!|sment procedure^ This was, of course, partly because is 
iiei^essaty, but also because the researchers, who were obviOTsIy not able to 
insist x>n certain courses being foDbwed, were obliged tb settle for an assessrnent 
model alreadj^ in O-grade one - because ftet^model matched, 

and, indeed, determined the^ sort of work the pupils were \^ing tb db in the" 
year. Ahhough several: te^^^ divi^ modjgr their S4 teaching slightly, by ex- 
cluding past papers or including mot? 'close reading', no schoal took up sug- 
gestions that folibsmight^^c^^^ of work arising from 
courses which weie not directly derived front the format of the O-grade exami- 
nation, e.g., a drama-based br rbl^^^ course^ or a course oh the 
press^ This fact is recorded without imputation of laziness or hidebound con- 
servatism: the pupils were, in fact, takjng the O-grnde examiriatibri arid their 
time arid their teachers' time were limited. 

It is arguable that assessment of pupils' performance in courses which varied 
greatly frbrri sch^^ as the marks might be 

measures of quite different achievernents: If this were so^ the criticism that the 
P^'oj^ct was trying tb do two s^ratei^d cbnfiisirig tilings, at tile same time 
would be justified. The ex^rience of 6SE Boarxb suggests^ however, that 
standardisable assessments ^ Erig skills exJbiibi ted ii^ di_fferent_ types of 
course: can be made. This project can make no contribution of evidence for or 
against that proposition. 



(d) Lack of time _ 

By far the rnost significant practical problem was pressure of time, both long 

and short term. 
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There was a general feeling, most intense in the Grampian jcgioii. that the 
preparatbjy planning short. Initial contact was made with most 

of the schools in late February, 1976^ for an August start to the experiment^ 
the Grampian schools w^^^ The papers describing 

the proposed scheme were circulated in April and departmental nv ctings to 
discuss them were held with the Research during that month. Agreement 

' was reached tlien that the 14 schools would engage in the project as set biit in 
Chapter n, and in June, when the first Trial Marking meetings were held, there 
was furlheridiscussion with the teachers about the procedures: Several teachers 
felt, these arrangements were hurried arid would haye preferred a full year's 
riotic(^ IlsJoujd,_hp_w^^^ noted that the shortage of planning time may 
have been more acute for some teachers than for others because of breakdown 
in cdmnlUnicatibri within thpir schools. 

Whether the internal assessmen scheme could have been better prepared by 
the schools in a year than in three to four months is debatable. A year's fore- 
knowledge v^ould not necessarii5' meaiv a year's preparation, because teachers' 
time is yery easily filled up by matters of immediate concern. It seemed reasori- 
abie to the research team to suppose that one_t_erm in advance is as much as a" 
depajLtment would normally allow for planning sucii a change, but perhaps a 
few months more, at least, woujd have helped teachers to feel more at; ease 
when the iriterrial assessment scheme started. 

Whatever^t^^^ allowed, there is no doubt that the teachers 

were very hard pressed to fulfil the demands of the project arid carry gut their 
"oi'mal teachiri of them did succeed in both, some 

spending many out-of-school hours in preparation of new teachtng riiaterial and 
ill assessing pupils' work. It is^^^^ and good use of the available 

time can make a success of an innovation^It would, however, be a distbftiqri 
of the truth to report othenvise thari that Eriglish teachers are carrying a very 
heavy: burden of class preparation, correction and assessment, so that any 
^additional work is really more th^ they can take bri without detriment to 
soriie^sFKict^ of theiF teaching. There is no consensus of opinion, let alone 
research evidence, iis to the time needed to plan Eriglish classes and assess 
wnttert work^ but there m_u^^^ beLSome doubt whether English teachers can now 
perform for all pupils all the functions they would regard as part of theii^job : 
they would then still be bver-cbmriiitted even if folio assessment tasks replaced 
some they already carried out, such as the marking of 'prelim' or end-bPterm 
AS pbssible tb bbtairi a rough impression of the demands on 
teachers' time from the following set of figures, which indicates approximatejly 
- the. amount; of time per week English teachers iri those of the project schools 
which provided this information can spend assessing the written work of each 
: of their pupils. 

In the course of normal English classvyork there is, prbperly^ a great deal of 
informal assessriierit^ a^^^^^ and. also properly, pupils^ will 

produce some written work which does not need or is tibt mearit for rriarkirig. 
Even when these points are remembered,_the figures indicate the immense poten- 
tial volume of written work an English teacher may receive each week for 
assessment outside of class contact time. 

'Average assessment time^ per week' was calculated^as the contractual 
hoursjjesscjaisiconta^^^^^ less two hours for class preparation. (Two hbUcs 
is a very ungenerous allowance for thispurj^bse, even if the teachers have man- 
aged to use sbirie j)ost^^^ free time the previous June to plan the * 
main lines of their work for the current session.) Mariy teachers, of course^ 
work at home for Ibriger thari the five hours per week which the 32* hour week 

' 27^ hours in school, 5 hours ai home: • - * 
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TABLE 7,1: TiME AVAILABLH FOR ASSESSMENt OF WRITTEN WORK 



i f^o. of \ No. of Pupils per 
ScttdOi 1 pupUs ] teachers . teacher 

per week 


A verage 
assessment 

per week 


Assessment 
timeper 
pupil, 
per week 


14 


' 1700 ' \\ ' 155 

j 


1 ^ 1 

7 hours 

420 mins 

(plus apprdx. 

i \ \ hrs for 

! travel 
between 
buildings) 


2J-3 mins 


21 


1050 ; 8-5 ! 124" 


6-7 hours 
= 400 mins 


3 mins 


22 


1700 13 131 


6-34iaurs^ 
380 mins 


3 mins 


23 


800 .7 J 114 

1 : 


7-6 hours 
= 455 mins 


4 mins 


31 1560 ^> ' 8-5 - i 183 ^ 


6-3 houre 
= 378 mins 


2 mins 

ft 


32 • 1700 !4 r2I- 


7-7_honrs ::: 
= 460 mins 


3J-4 mins 


34- ' 1150 8 ! i44 

_. - — i — -"- ■ - i 


7:7 JiOjLirs _ _ 
= 460 mins 


3 niins 


35 = 1300 jO-5 124 


6j hours. 
= 378 mins | 


3 mins 


These 8 10960 80-5 136 \ 
schools ^ ! 


jipprpJ'- I 
7 hourb ; 
420 mins s 


3 mins 
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requires* and sb^^ departments can perform organisa- 

tional miracles to use time more efficiently: Table 7:1 seems to suggest^ nevef- 
thelesa, a genuine ne^^^^ To*" considerably^ m^^^^ teachers, 
if it an important part of their work to assess written work either for diagnosis 
of st»^engths and weaknesses or for discrimination. 

An alternative or additioi! ' ameliorating step would be to reduce class sizes 
further arid so also the vblui written work per teacher. 

While it' is not the _4>usiries^_,/v this p make recommendations about 

pupil-teacher ratios, it is legitimate, in the context of discussidn of the practical 
Pl^°*^^^*^s felatiri&^t^ order to allow a 

teacher to spend J?ve mimites per v/eek assessing each popil's output, within con- 
tractual working hburs and with the average f^^^^ the eight schools 
in Table 7^], the pupils taught by each teacher per week would have to number 
about 84 rather than about 136, This would reduce all classes to about 18-21 ^ 
Pi*i5ijsia_nd wpu Id -require the employment of nearly 60 additional teachers in 
these eight schools: 

/' There is at present a natural tendency for teachers to concentrate their 
marking effort bti certificate ci^^^ The JnipIications for 'certification for all' 
of English teachers' shortage of time are clear. 

As far as (he project itself concerned!, it seojis imvitubie that, without staffing. 
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9 ^^^K^^Wj ''I /''A' MUWJW'^ ^f^ouh t of work to he c^sessed sfiquTd he 
rccomfnended for any future internal scheme, though enough should be im^uded 
to ensure that a range of different types of Work is covered. 

Despite the yyrohtems described above, it is not the researchers^ opThTon that inf er- 
naJ yissesymeni^ because^ of Jpressure of work on teachers. Some of the 

project schools, and many in England, have shown that an internal assessn^ent 
Programnie caiviwdrk ycjir welL TTi^ ifnp_a^^ shdrtage\oF 
tinse ii tliat it appiies whether or not assessment for certification is internal; 

ffi^ English teache^^^ almost impossibly hea redueecb 
it may well be possible to change some of it from direct preparation for and 
practice testing •for the external exam iriatidri to assessment of iriteiTially de sign ed 
courses. The project schools were abnormally heavily burdened by having to 
do both at once. 




CHAPTER X 
TEACHERS' VIEWS, 1977 

L The Questibrihaire 

The 1975 survey of the viCws of over 500 English teachers produced the follow- 
ing idiyisjori of answers to the question whether they would welcome suitably 
moderated internal assessment instead of ihh O-grade English examination. 

TABLE 8.1: DESIRABILITY OF INTERNAL ASSESSMENT (1975) 





No 


No Response 


51% 


42% 1 


1% 


100% - 523 English teachers. 



This s u rv ey was ii . . ide in 1 04 s^Hpo Is, one jn/o u r of those ^ reseri t ihg carid id ates 
in the O-grade Englisii examination. It was not possible to know precisely the 
total number ot English teachers in these schools^ but it was reasonable to guess 
that the figure of 523 represents 60% - 70% of the total possible sample: 

At the end of session 197^77 the teachers who had participated in the internal 
assessment scheme were asked to fill in a questionnaire and the researchers 
visited each school to collect t^ and receive^ariy word-of-mouth 

comments: Seventy-three of the 114 teachers Involved completed at least some 
PArts of the questionnaire^ though they form a very diK sample from the 
1375 group, their x>pjnions are of special interest because of their experience. 
Tables 8.2 and 8.3 show t sections of the questionnaire 

which listed possible advantages of coarse-work assessment generally, and of 
the 'rules' of the SCRE experiment in particular. 

TABLE 8.2: ADVANtAGESL^^^ WORK ASSESSMENT: 

TEACHERS-V VIEWS (1977) '"^ 



Advantages 


\Did not occur , \ 
' //; project. Occurred. \ ■ 
r Occurred hut a hut not Did not j No 

potential regarded as occur and 1 response 
advantage ^ an advantage \ HinTikeTy j 


Wider range of w* rk 


14 (19-2%) i 26 (35-6%) 1(1-4%) ' 27 (37 0%) j 5 (6-8%) 


Work belter suited to 
pupils 


17(23-3%) ' 17(23-3%) 2 (2-S%} ' 24 (32-9%) j 5(6-8%) 

! 0 ■ _ _. _ 


Pupils beuer motivated 


33 (45-2%) ; 13(16-4%) , 0(0%) 20(27-1%) 8(11-0%) 


Work of better quality 

» ..... 


18(24-7%) 1 15(20-5%) | 0(0%) ; 30(41-0%) 


10(13-7%) 


Pupils more accurately 
assessed 


24 (32-9%) i 22 (30-1%) { 0(0%) 26 (35-6%) 


1 (1-4%) 


No Jjjne iosl_for 
*prelim^ examination 


9(12-3%) i 5(6-8%) ' 7(9-6%) 42 (57-5%) 

1 i 


10 (13-7%) 


Better course planning 


IO(j3-7%) 28 (38-,!%) ! 0(0%) ■ 24 (32-9%) ' 11 (15-0%) 


Useful teaching 
material prodticed 


34 (46-6%) : 13^-8%) ; 5(6-8%) 12(16-4%) ' 9(12-3%) 


Teacher more aware of 
standards and tech- 


38 (52-1%) 15 (20-5%).! 1(1-4%) ! 12 (16-4%) 1 7(9-6%) 



100% 73 teachers. 
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TABLII 8.>:^\DVAN TACil-S ()l SCRE PROJECT: TEACHERS' VIEWS (i977) 



Small Definitely 




rroicct Ktement Very Acceptable 
acfvan- 
tageons 


1 advantage not 
only \ advan^ 
tagcoHs 


! Undecided ' No 
• response 


Defihitibitorairiis, 21(28-8%.) :^U38-4%) 
as pnriciple . 


' . 1(1-4%) 2(2-8%) 


S > ' 16(21 9%) 


Actual aitns agreed 14(19-1 %) j 29(39-7%^ 


j UI-4:%) 1(1-4%) 


7(9-6%) 


21(28-8%) 


Specification of 22(30-1 %) 23(31-5%) 

mininmm work, as 

principle 




9(12-3%) 


15(20-5%) 


Actual minimiiDi 1(1-4%) ' 24(32-9%) 
agreed 


1(1-4%) fl(l5-0% ) 


15(20-5%) 


21(28-8%) 


Take?; iiccount m 19(26-0%) . 18(24-7%) 
improved work 


6(8-2%) i 8(11-0%) 


7(9-6%) 


15(20-5%) 


Take?; acctnini of 19(26-0 ;;;) ! 25(34-2%) 
best work 


4(5-5%) 4:5-5%) 


8(1 10%) 


13(17-8%) 


•Trial marking' • 18(24-5 %) '. 29(39-7%) 
exercises 


13(17:8%)! 4(5-5 %> 

i 


9(12-5%) 


0(0%) 


iVIoderator's visit to 26(35-6%) ' 17(23-3%) 
discuss levels of 
difficulty of work 


6(K ->%).. ; 2(2-7%) 


17(23-3%) 


5(6-8%) 


Moderator's visit to 25(34-2%) 22(3() ! %j 

discuss standard of 

marking 


4;5'^% 3(4-t%j ! 


10(1.1-7%) 


9(12-3%) 



100% 73 teachers. 



Pescriptioris of the aims and other elements of the project referred to in 
Table 8.3 can be found In Chapter II. The list iii Tabic 8.2 consists of ac\ antagcs 
rheriti pried by teachers whcri the research team Ha J visited the schools in the 
coarse of the year: In discnssioji during these visits it was noted that some 
teach ers saw likely ladvari tages: o f [ n tern aT assessment whi ch \vc re ri o t , hp wc ver^ 
manifesting themselves in the experimei»*q:i scheme becaasc of its 'teething 
t f ou bles' and because of the const rain t.s of ti me eaii sed b^ halving jo iprepare 
for the O-grade examination as well: 1 1 was for this reason that a category of 
'potential advantages' was ificliided in the questionnaire. 

ii is viear from the whies tJiar , wrfwn they ure vonsidermg tire eztucziiioimi a^- 

among these teachers is favoiirabie to iniernai assessment and to the form of it 
^i^Pfl^^L^y ^'C/?^ /7ro/m Tn Table 8.2, responses in the /Occurred 
Totcntiar columns outweigh tiiose in the other two columns in, every case, 
except 'No timb lost for "prelim" examination', and, in Table 8.3 all thejisted 
eicmcnts of the project received more sUpport than criticism. U is noticeable 
in Table 8:3, however, that a much larger proportion of teachers made no 
eomriient ori some particular aspects of the project ^ especially * Actual aims agreed - 
and •Actual minimum agreed': Theyjnay have felt, as the researchers did, that 
it vvas difficult to judge the value of some of the requireriierits of the project 
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in ah experiment-lasting only two terms and carried out under the pressure of 
the need to prepare pupils for the external examir'itiori. 

The-imprej>sion that 'he majority of the questionnaire resppnderi favour- 
ably ^ispQsed taroHp assessment is reinforced by consideration of their reactians 
to most of the items which invited judgments about possible drawbacks. -Table 8^4 
shows that m not suffered or^ m a smoothly 

operating scheme, would not suffer from the disadvantages, which, again, had 
been suggeste'^ tr Ihe research teAmib^ the course of the cxr 

perirnent. Th' . clear disadvantage was the- tendency to concentrate bri S4 
^he 6 rirnent bf'bther classes^ large numbers who found 

or envisaged dilficuity in keeping folders in act and in oTder are nofewbrthy. 
These are both management pfbblems rattier than educational issues. 



TABLE 8.4: DISADVANTAGES OF FOLK^ ASSESSMENT FOR PUPILS: 
TEACHER'S VIEWS (197/) 



Disadvantagi^s \ Occurred 

\ 


DTd hot occur 
Th prdject, 
but a 
\ pdlehttat 
■ dtsadvahtage 


O c c u r r e d, b ut . — 
hot regarded IDTd not occur 
as a [ and unltkeiy 
disa d va n t ag e | 


No 
Response 


Too much writing for j II (15-0%) 
pupils 


j 17(23-3%) 


7(9-6%) ' 34 (46-6%) 


'^(5-5%) 


Too little discussion 8 (11-0%) 
for pupils 


: 5(6-8%) ; 0(0%) 51 (69-9%) 


9(J2-3%) 


Lowers standards j 3 (4 1 %) 


9(12-3%) 


0^0%) r : (74 0%) 
1 (1-4%). ! 4? (57-5:',;) 

i 


7 (9 6%) 


Made it difficult to J \Z i\(^'4%) 
prei3are for O-gradc ; 
this year I 

i 


' 9 (12-3%) i 


9 (n-2rM 


RjLipils less willing 2 (2-8%) 
to work j 


; 2 (2-8%) j d(()%) j 50 (68-5%) 

; - ! j 


19 (2t '^"o) 


Uhhcccssary anxiety ' 4 (5-5%) 
caiiscd to pupils 


I4(l9-j%) I j (i-4%) ! 47 -^64-4%) 


7 ^ 
3 14-I /o) 


Ovcr-cdhcehtralibh 22 (30- 1 %) 
on SIV by teacher , 


23 (3i-5%) '0(0%) ; 25 (34-2%) 


Folders difficult to \ 17(23-3%) 
keep in order ! 


14(19-1%) j 4 (5-5%) j 34 (46-6%) 

1 

i 1 


^'(5-5%) 



!^% = 73 teachc- 



The items listed in Table 8.4 were not "differentiated as 'Disadvantages for 
Pupils'^ in the leachers' qliestidnhaife, but were part of a longer list which 
included the five items in Tab!>3 8.5, now labelled 'Disadvantages for Teachers': 
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TABLE 8.5: DISADVANTAGES OF FOLIO SMENT FOR TEACHERS: 

TEACHERS* VIEWS (1977) 



r^sadvaiUages 


1 

! Occurred 


Did_ noi occur 
in vroJect , 
.buta 

. PQj enjJqi 
disadvantage 


Q. cur red. hut i 

not regarded iDJd noi occur 

Qs_a_____ \ and unlikely 

disadvantage ; 


islo 
response 


Too much marking (of 
normal classwbrk) 


21 (28-8%) 


15(20-5%) 1 Id f 13-7%) 


22 (30-1%) 


5(6-8%) 


Tod miich' planning 
time required • 


12(16-4%) 


14(19-1%) 1 10(13-7%) ; 30(41-1%) 1 7(9-6%) 


Too much Mrial 33(^5-2%) < \S mSl^) 1 10 (13-7%) : n(I5-l%) i 4(5-5%)- 
ivfarking 


Too much 25 (34-2%) ! 21(28-8%) ! 7(9-6%) 
admfnfstrativc work _ 


H (15-1%) 1 9(12-3%) 


Rroblem of finding I 30 (4M %) 1 15(20-5%) ' 12(16-4%) 
leaching material 


n (15-1%) 1 


5(6-8%) 



\60% = 73 tcijhcre. 



jn arisweririg this last group of questions is the majority of the teachers 
found to regard the fblio assessment schcnic unfavdurably. Concern about their 
P^*^jMr£ jSarkinj was heightened for many by the' extra obligation they 
feU the project imposed to pay close attention to a// the folio work, even though 
it had been agreed that ^ at the time it was done. 

The fact that a moderator might be looking at the folios probably helped create 
^^^s feding, as well as pupil and j^arent p>esj^ The Trial 

IVfarking' exercises involved more Assessment work and a.considerable amount 
.of adniinistra)tiv0 work; tob^ all^^b^^ with 'the final as^essm^^ 

the Tpjid and the recording of scores, overioaded the teachers; Many of them 
felt, too, that the research team's suggestion that they sjibuld riot use past 
S^perS: as teaching material added considerably to their burden. The resuli of 
so much pressure of work was that, despite the perceived educattonat and pro^ 
f^^^^P"^l A4^^^^ of the scheme, a majority of the 73 respondents said they 
would not welcome a similar scheme to repiace the d-grade examination and more 
^^/^^/^^ /^J^^ *^^ tnigKt be n aJtcrnative to the 

. external examination: The figures are shown in Table 8:6: 



TABLE 8.6: DESIRABILITY OF FOLIO ASSESSMENT (1977) 



^ I . Folio to rei i '- v O-grade 

n j5-i%) 

2. FoKo as alternative to O-grade 

Yes ----- 
29 09-1%) 



No 

42 (57-5%) 



30 (4M%) 



No response 
20 (27-5%) 



No response 
14(19-1%) 



100% 73 teachers. 



: The questionnaire enquired aJso abant thc-cxtcnt to which the projcLi in- 
fluenced the course taught by teachers in five schools 
said they had not used past papers when they woald normally have done so, 
and a further 15 in these five and other schools set less past parcr work then 
usuaL FoLir teachers omitted some other work they would other >e have done: 
in one case this was a test on *thc nover and the other thre- rorii the same 
school, said: tMl Uicy: had felt an obligation to get pupils iting to fiji the 
folder to the detriment of general language work and oral i k. On the other 
^^"^v 20 teachers felt the^^ set new types of woji .cillcally. because of 
the: project. This feil into three -categories: (I ) u widci uingc of writing tasks 
^^V"^ s^'"^"li, (2) a wider ra^^ interpretation passages, (3) more 
'close reading' of literary texts. 

H. Coihinehts 

T'lc views of teachers wcT^^ in the course of the experi- 

rnent, by invitation to Principal Teachers to wj-ite to the research team, arid 
also, informally, during Trial Markirig^^xereises and the researchers' and 
moderators' visits to the schools. The contents of T-ablcs 8:2-8.5 may be regarded 
as indicating the range of matters of concern which cariietb notice in these 
ways, since almost all the listed advantages and disadvantages werementioned 
by the teachers.: The answers to the last qUestidririaire item, inviting free eom-: 
nipiit perhaps show w^^^ issues sprrie teachers felt strongly enough to expand 
OTij'Almost iilj the: statements made arc given below: those omitted related to 
^spectl of the i>rq^^ on the desirability of course-work 

assessment, such as the nature, length and timing of the Criteridri Test, or 
s™p'y. repeated a ^omm elsewlicre on the ques- 

tionnaire. A few remarks made in answer to questions earlier in the questionnaire 
arc included here with the final comments. 

'-Numerically, the statemcrits ean be diyid^^ the followjiig headins^^^^ 

LTha groupings are, ho\\ ever, sonicv hat arbitrary and several comments make 
more than one point;) While the v rii^bers of favourable arid urifavburable stat^j- 
hi^ts give u _roi:gh iridication of th*: issues causing most concern. It should be 
remembered that the totals include niorc than one coriiincrit by sdriie particular 
teachers. 



NUMBERS Oj FA VOVPABLE COMMENTS 

relating to: 
Assessment and course flamming 

PUPIL MOTfVATION 
TEETHING TH'.'U! }^tES* 



7 
6 
3 



Total 



16 



NUMBEHS a/'JlNFA VOVRAinM COMMENTS 



RELATING TO: 
OVERWORK 
?A%T PAPERS 



6 
4 
3 
6 



PUPIL MOTIVATION 
FOLDERS 



MISCELLANEOUS 



Total 



30 



Favburdhie Cbhwiehis 



Assessment and Coarse-planning • 

Hcjpiiil for internal I'sscssment within the dcpartnidnt. (Fife 4) 

The exjDcrimejit proved 3^^^^ in ^hat a dcpartrricrital roriirtv for discussing 

and jastifying^ marks was set up, and as a result I feit I examined my own 
assessment of material far more closely than I had ever dbiic previously. 
(iSrampian 4.) ^ 

In general I would say that the folio system is fairer to pupils by eliminating 
the *ldttcry element' of the exam system, fn addition potential Higher can- 
didates arc not restrained by the demands of the' 0-grade exam. (1 know 
• ^this shouldn't happen, but it does:) (Grampian 4) 

Wider nrngcof creative writing tasks and moradetaiicd close rending of areas 
pf texts J*^: literat^lr^e wJ written answers. (Work done in 76-77 dilTerent 
from normal:) (Grampian 2) 

Top-flight pupils can develop more quickly and are not restricted to G-grade 
work for most of the year. (Gfampiari 2) 

Iri general, continuous assessment and internal assessment arc worthwhile arid 
could be used as part,' at least, of the overall assessment of pupils for O-grade. 
Teachers of English are niude^morc aware of the aims of an O-grude course 
and the problems of asscs.sment. A single assessment of the completed folder 
docs, however, seem a bit inadequate: (Grampian 2j) 

■ The production of original material (for interpretation etc;) is a potendal 
^Ldvant^i^^, if done in an organised systerri^ allhough it is sometimes hard for 
individual teachers to ensure a suitable standard of questions etc. (Fife 3) 

Pupil Mbtivatioh 

Main advantage w^^ to provide a useful motivation to less good O-grade 
classes (Fife 4) 

Increases amount of work done by poorer pupils because of greater moti- 
vation. (Fife 4) , 

Class has improved on ;mi all-round basis of about 20/.25%: 1 feel there has 
been ai greater app licat i on to cl ass; work: ^inccj th c fi rst : assess m t- n t . C | a ss h as 
learned to apply itself with greater concentration; language .^nd word sense 
llfis been sharpened to a greater degree than iprey ipusljy^ The main jmprbve- 
ment has been mechanical/technical in the fields of language - greater spelling, 
Punc^Mli^iph, gram I feel this project has helped the class 

tremendously. (Strathclyde 5) 

Advantage that there is a requirement placed on pupils to work through the 
whole year. (Fife 3) 

classes jikcd this method of working f^lt It fairer: Everyone felt they had a 
better cM nee an^ rushed. I myself feel it is a fairci system and woiild 

like to see it adopted on a national basis: 
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I found this scheme :an advantage in that it would force pupilsi^tb work 
thrdUghbUt the^^ear. I find:^ those pupils who don't try aii 

year and then manage, to 'turn it -on' for the exam. Also it helps those who 
are *bad examinees'. (Strathclyde 2) 



'teething Troabjes' 



I would sayihat the staff are now reasonably satisfied^with the operation of 
^'^^ project. Mps^ problems and uncertamties have been cleared 

up and the staff are attuned to the process involved. The stresses and^strains 
of day-to-day teaching c minor problems. One of the stresses, 

of course^ is caused by class exams which go on regardless of this prxyect 
^d which inevitably, iri the settingi^itting and marking, take up a sizeable 
chunk of time in each term: (Grampian 4, December 1976) 

LCound it a little difficult running this scheme as well as the O-gfade. I think . 
that if used on its own it could really work. It was difficult to avoid using 
past papers because it was necessary to prepare for the G-grade itself. (Strath- 
clyde 4) 

Generally, i approve of the scheme though it does place biirderis^n the less 
abl • pupils and their teachers arid, there was/a great deal of administrative 
work involved. (Strathclyde 2) V / 



Uhfavourafyle Comtnehts 

Overwork 

ampurit of stairwork impJied'^*4iuge, given that all teachers will Jmve 
something like:4 other classes, Iikel>; to include one H, arid possibly ari SYS. 
Tweilty-orie substaritiaj corrections in a session for one class must affect the 
others. N.B. Three of my staff had mo S4 classes in 1976-77. (Fife 4) 

Moderation and final cross-marking takes vast amount of time. (Fife 4) 

-Lwould ^yelcbr ction of a folio scheme of assessmerit.' ... not 

unless paid for . (Fife ^) 

Life is too she icularly when dealing ./itii tWo fourth year classes. 

Other clasL-^s' > ured. (Fife 4) 

T'^^^n^^rc project has been ver^ tjriie consuming and tended to become rather 
tedious at times when preparation-correction time was at a oremium. (Gram- 
pian 2) ' . 



Tremendous amount of extra cbrrectibri arid extra paper work. (Fife 4) 

Single assessment of folder requir d after hurried niarkirig of 4th year prelim 
exarils, 3rd year arid 2nd year ex^iiis. (Grampian 2) 

^i"^*"."*" requ^^ to be possible to adhere to 

the minimum requirements with the bottom set. These requirements are 
already nearly exceeded by the top set. (Grampian 4. December 1976) 
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Willin^ess to produce second version or third version increases pupils' work- 
load. (Fife 5) 

The mp\n criticism of this particular_exj5eriment was the lack of time; More 
t L^^IQ^ n . • ght ha yc: been a I lowed fo r t he d i sc u ssj ori: p f t h e_ sehem e bef d re j t yvas 
intrnJuced and sample materials should have been available to the teachers 
ir'volved months earlier. (Grampian ?) 

' ^ b 1 cs I c r r 6 1 i o A ssessm eri t ' agreed ! ast s u irime r d id riot a rri ye u ri ti I 
::.turnfi when »be session's work was w?»lj ynder way - therefore \ paid 
or ice to the rule-. jr;t' while writirig wa?^ well represented, reading and 
■ctjrpretatiqn were undcr-ropiestritcid. 

I felt my coniribution was negligible since the scheme was presented with 
'^o time to ^ve it adequate preparation or pranning. My froup contaiiis-a 
^: 'h propprtiph Higher candidates and I did not find it helpfu! 

xo be instructed to give a student 30/30 because it was; top C Trade standard: 
The scKe me see med to h ave Jwg aim s : {a} to stan dard i se t each cr assess merit 
(b) to produce a wider range of teaching and testing material .These aims 
vvx-'vUld have beeri better served had they beeri treated separately. (Grariipiari 2) 

(fSbte bri the la.st two cbriirilertts: the cfiticisriis made indicate sbnie bf the 
problems of communication of instructions and advice. In fact, the material 
referred tb was sent tb the school in April, fbr an August start tb the experi- 
ment, and was discussed at two ineetirigs with the department in April and June.) 

Past Papers r ^ _ 

!iTiJ>0ssjbJe ript: ta use past papers, as rib substitute is dlT^red or sugps^ 
If teachers were responsible for producing and preparing interpretations etc: 
Pf tJlLei]" pwn t^ ^pMd_ have rib tjriieja teach ! Ariyway what is wrbri^ with 
using past papers discriminjiicly? (Grampian 4) 

Work done in S4 academic classes must be seen as preparation for O- and/or 
H-grade exams. Where pupils hope bf expect tb pass these exams, teachers 
will be failing in their duty if they do not teach for these exams. Defenders 
of^the Scheme will doubtjess claim that it provides adequate preparation for 
SCE exariis, arid sb iri thebry it may dp. But in practice it cannot^ at [east as 
originally envisaged. And using the Scheme only in part or in watered -down 
rorm can prove ribthi rig, while it will place an extra burderi on teachers iri 
marking and administration: (Grampian 2) 



At the moment it is essential for the pupil to succeed in the - "^-jrade 

exam as \veil as having, his standards irilpfbved. (The fofmef dr . ;:eccs- 
sarily follow from the latter.) Formal examination practice, sue! ;i > pro- 
vided by past papers and Pillans & Wilson, is necessary fbr this. (Stfathclyde 5) 



\ fail to see why past papers cannot be used for class practice in addition to 
c^her irit^rpretatibn vybrt. The ampurit of preparr^tibri and_ cprr^ctiprr tirfie 
sometimes makes it extremely diificalt to prepare '*qnestions" for the pupils 
- thisis a purely practical pbirit!' (Fife 4) 

Work bmitted because involved in this scheme? - Past paper work, which 1 
feel is very beneficial (since they still have to sit the O-grade exam). (Fife 3) 

FvvvvT interpretatibris done - difiiculty of making up original tests in sufficient 
quantity: (Fife 3) 

«9 



Pupil Mbtivatioh 



PupH motivation ^1^^ to gauge; There was in fact tile average crop 

or absentees, disciplinary suspensions, failures to hand in written work which 
suggests that one awkw^a_rdlelement of the population was no more impressed 
by thij than by my usmi demai^s for a fair proportion of written work as 
a condition of entry. (Pife 5) • 

The pupils involved did hot take the project seriously. The impact of exams 
was completely lost and much of the written work demanded was not done 
It was quite impossibles to get any work done at home. (Fife 3) 

The scheme in itself could work, but only givea pupil^ who respond: The 
group 1 had were, jx^rhaps, the poorest working group J have dealt with It 
wasjilniost impossible to make them Undertake the assbssments set and the 
standard of work suffered,. An exam (with the less mouvatcd pupil) has an 
imj>aeX and means much more than any other viable alternative I have vet 
seen. (Fife 3) 

Lack of Preiim/Pfelims lessened motivation of pupils. Folder-vvoric.soon lx3St 
Its importance for poorer/less wijiing^pupils. Lack of timetabled Prelim left 
pupils short of 'exam' practice. ( Fife 3) 

Folders 

Loss (of folders) due to lack of classfooni security. (Strathclyde 4) 
Folders make revision difflcujt: (Fife 5) 

Need for jotters. The stiiff felt it was necessary to provide jotters for rough 
work^and for some exercises. The folders received by the moderator do not 
^lclude thes<^Jptters^ which do in some cases contain a substantial amount of 
• work. Presumably suitable work in these jotters should be used for assess- 
inent work although up on material in folders 

(_Fhere was no real justification for supplying a jotter other than the teacher's 
deep-felt belief thac a pupil iieeds a jotter!) (Grampian 4, December I97r>) 



Miseeljarii.bus 



03nsiderable:^ukk^ required tor marking - I don't knoSv how. The notes 
marking compositions arc very helpful, but lengthy. One copy can also 
CiL-'ly be lost or mislaid. (Strathclyde 5) 



f found that there v/as greaterp:re!ssure to produce a stockpile as an end in 
itscjj. hi t ic normal year this stockpile emerges as a natural end-product of 
cJass woiv. ! fc!t that my olycctiyky:, m^ own capacity to assess: soberly and 
thera^cuaco tJy, was slowiy eroded by thc^ endlessness of it. F felt also that 
the ar! v:i rnap.t N m motiVatmg some were oJTset by the tantrums of others who 
l<ythc\nrs_t t:irne actually expressed their frustration and sense of inadequacy 
with co:itin«mg failure: 

There IS a final conflict: are we assessirig-critirely independent of O-grade 
prognostications or arc Ave trying to match ourselves against the O-grade as 
some ^ .d of sacrosanct absolute? Although the idea is clearly to replace 
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the O-grudc, this year's cxpcrinlcrit has been bedevilled by the necessity of 
trying to foretell accurately how the candidates will actually perforrn: not 
the same thing as selecting the highlights from a long period of work. (Fife's) 

Nattonaj examination is preferable: (Fife 3) 

I have a dislike of any scheme ol internal assessment, as opposed to an out- 
side exlirii, since it involves too much of the personal. We live here, arid meet 
frequently the parents of our pupils. Examiners arc faceless (? froze. j kippers) 
<»kid opprobxjum attached to theni is acceptable. Iri the small cdmriiuriity, it 
would not be, despite the interven tion of moderators - who, for ' most 
part, arc also our friends and colleagues. (Grampian I) 

Single assessment-allows no . is: rccdback to pupil: Gives icachcr no 
cbntiriijirig idea of prbgre^ ^'.'^! Moi uf I) 

" j ^ ^ ^ ^^^^^ ^ ^_"^b u raged i> ■ i u U ;t 1 i ri i ti a t i ve , but m i gh t ri ot h a v c this e ffect 
III liic iong-term— merely providing a set of hoops of an initially difVerent form 
which rillght well becdrrie as starid:ird as those they replace. The rerrlbval of 
certain ^props' might have an advcrse-efTcct by reducing general conridence in 
one's own teaching - even the reversion to the odd *PiIlaris & Wilsori^ with a 
threat that it mii^ht be used \\\ evidence, has a consoling effect on occasion. 
(Fife 3) 



III. Some Recbmihehdatibns about Practicalities 

Regarding practicalities and teachers' opinions only,, it would seem that the 
following would be desirable elenu-nts in any future internal assessment scheme: 

i; There should be adequate planning time - perhaps one year: 

1, Daring this time thcre shoald be planning of course aims and Trial Mark- 
ing exercises to *teach' assessrrieiit techniques to tire teachprs 

3. Sbriie ways should be fouiid to reduce tlie tiriie cblnitiitrilerit of teachers 
when the -scheme is in progress : 

c:g:: (a) Reduction of the minimum number of pieces of work for 
assessment I'ronv say 4cbmpbsj^tibn.s^ ? /tcsppnlsiss 

to fciteraturc' and one on two interpretation tests comrnon to 
tlie whole school. . _ 

(bj Schbols should iKive ht>lit) Assessinent of Ihterhal Exann'i - 
tion, not both (because of the imw involved): 

(c) Serious consideralibn should be given to the provisicn of r-ic-. 
free tirne for Englisji teachers for assessment ancf/br to 
reduction of class sizes to reduce the load of writtci v^ork; 

4. The favourable motivation of teachers should be sought by / 

(U) Giving iriternal assessmer t significant weight if there is also 
an external exainihation. 

(b) A shift of emphasis as to the purpose of assessment iiv English, 
to ericbUrage greater cbricertl 'for description of a pupil's 
strengths and weaknesses, which is better done by teachers 
than by external e iniiiers. 

(c) (Ideally) the prbvisibn bf sbriie payrrierit for internal assess- 
ment for certificuetion. » 



^-uctiohs should be clear and simple with regard to 
i) the purpose of the assessment. 

(b) what is being asse"ssed (i.e., achreVement, rather than ability). 

(c) the method of recbidirig assessments o"" 

Each school ijivoived should delegate respbrisibilijy for the implementa- 
tion of internal assessing member of the English department who 
IS committed to the method and has the time, energy, knowledge and 
^_°^P"^:^^ ^"cojirage others,: arr..ni;e jacetings, give advice on standards 
witjim_,th^ school, inir uc dcc:.i::;^:. insist on adherence to instructions 
and engage all m Trial \,v^ ; ^'vfdderation procedures 
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CHAPTER XI 



* CONCLUSIONS 

SC*- • project Was Undertaken inj>ursuit of the following aims, which are 
also set out on the first page of this report: 

I. To report on the present practice in assessment by schools of candidates 
for Ordinary, Grade English. 

To investigate a^^^ in which teachers can 

make assessments (possibly in the form of orders of merij) of piiji] 
perforraance in sdected asp^^^^ being based 

bri a wide range of work, over a substantial portion of the school year. 

3. To determine the best ways of scaling and/or mdderatirigjhese assess- 
ments so that assessments of pupils iri different schools may be com- 
parable. 

4. To investigate such other rel^ated matters as wy appear in the course 
of the investigation to be relevant to the field of interest covered by the 
Board's remit. 

5. On completion of the above steps to make recommendations to the 
Board' on whetjier: alt^^^ means, utilising teachers' assessments, 
shouki be substituted for the whole or part of the_ jjresent O-grade 
examiKatioji in English^ ta outline the administrative changes that would 
bp mvolved. Costing of these changes might also be undertaken in co- 
(^era' on with officials of the Board. 



The iindintgs of the project cannot caslli he i-epbrted in direct relation to 
each of ^ aims, because^ as the research progressed, some of the 'related 
matt-rs re: .red to in Aim 4 were seen to be^b si^ificant in the evaluaticn of 
interna! ;i .>pme^^ with external that they assumed major im- 

portance. ^ rincipal among these significant 'related matters' are: the well-known 
prbblem < . marker jinrejiabihlty i^^ assessment of English; the tendency for 
markers to fail to spread their marks out widely (which may be aj-eflectipri of 
the nature of English aMMitfe actual dist 'English skills' In the 

pbpubMon); practical difficulties in implementing internal assessment,'j;aused 
mainly by severe constraints of time^^ridiheireactibn to internal :assessment of 
the: English teachers involved, which ranged from enthusiastic commitment 
through indifference to strbng hostility. Cbhclusibris are also iniluenced by 
another related quest ipm that of the relationship between assessment procedures * 
and the courses followed by pupils. 

The researchers w^/e not breaking new grburid in discovering that 'the assess- 
ment :bf English in Scotland is fraught with tiie same^ problems as are met by 
teachers and examiners elsewhere, and most of what^they have to report about 
tMrd|xpeflenee:btars_clb^^ to the findings of other researchers 

into English assessment^^uch as the authors of the various Jbint Matriculation 
Board reports oh theji:;GCE:P level Alternative English Language assessment 
and the Schools Council reports on CSE assessment. 

The SCRE project was in some respects a feasibility study in Scotland for 
a type of assessment already employed elsewhere fairly successfully and the 
eharactenstics of which are widely known. The principal significance of the 
present report is probably that it brings to light in a Scottish context various 
aspect^ and j)roble^s of the assessment of English, and serves as a reminder 
that they apply as forcibly here as elsewhere. 
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: . In two respects; however, the researchers bch'eve that they can cLaiifl'that the 
^'^'^E prt>ieet has been iiniisual. The presentation of the tables and figares in 
the statisticai evaluation of internal and external asscssmcht has been dejibcrately 
^'-'^^gncd where pbssib^^ factors as marker u ireliahility 

and the elTects of standardising and/or seahng raw scores on particiilar pupils; 
"^^''^ ^>^"IP^'*'^^^" ^^^^ rates, it was felu does 

not always lead to a clear realisation of the absolute numbers of pupils afTectcd 

?*'^^'*^P^"cics I'^i^twcen markers, t)r s^ bgujidaries by a 

^c'alinJj>rocess. The second unusual element in the research is the testing of 
the frequent assumption that a schoors rank order is likely to discriminate 
^riiPM Pypilsj hior^ vaHdly than that produced by an external examination of 
normal length: the Criterion Test allowed compansdn of bbtli with a-t4ii;*d 
"^cUsure which w^^^^^ reliable a test of English as coir - 

be put together and administered in reasonable time. The prbcess of making 

f^".^cridn Test, which i^^^^^^^^ of English tasks and skills, 

has incidentally provided material likely to be of value hi developing diagnostic 
or profile assessment in English. 



" ^'^^'^^ ^A^^V^M^^'^^y ^'^^ relating the researcheii' RncHngs to the fprmaily 
sta;. / aims of the project? A re-statement of the aims (without distorting them 
^ r. w?;;cM) in the form of live broad questions seems helpful. 

_ _ J _ 1 * 

1 What are the advantages and disadvantages of retaining unchanged the 
ex' -ting O-grade English arrangements? 

2. What gains and losses would ensue if 0-grade English were assessed 
wholly internally? 

3. What are the advantages of combined internal and external assessmer 
-4. Which methods of moderating standards are most cfTective? 

5- :What should be English teachers' rnam concerns? 

The O-grade English examination has many advantages: Its administrative ar- 
range m en t s a re we 11 - 1 ri ed an d e f\ ec 1 1 ve, ah d i ts^ mit i on wi d_e c |J rrency en s u re.^' 
eomparability of standards for pj.^, from all schools; in tii is ' respect *herc is 
particular advantage in having u^siuglc cdmrridn interpretation test, TheLre is a 
general satisfaction with the O-grade cxamin itlcn among both teachers and 
othci joscrs of its results. It is, indeed, very prdfe.ssldrially .set arid i.s prbMbly 
fi-"^ V'^li^^ ii^ P^M^^^^^^^^ year. Careful consideration has been given over the 
years to .the Hvashbac!'* effect on the school curriculum of the H^rni of the 
'^^^^^''^^PPHriP tha^ pHiipiflc, teachers : free to teach a wide variety of 
courses preparatory to O-grade. (There is evidence that in pritcticc, hbweVer, 
niiin y teac h e r .s d d n o t take ad v an tage of t h i s . 1 1 eed o rii an 4, m ak e ex ten s i ye use 
of O-grade past papers as teaching materia, in S4.) A farther credit to the 
JTxamination is the oppdrtunity It afTords td sdnie carididates to show their real 
>v<>rth- j which they niay have lacked th*e motivation to reveal in school work: 
this benefit is, of course, offset by the di.sadvantage .r pupils who work well 
in school but are 'bad examinees*. 

The more tm sat is factory charactciistics of the G-grade cxaminatfdr. besides 
liri y u n d cs i r ab le i ri fl u en ce oil te ac h i ri g, arc t h r ec fo 1 cl : 

( a ) U [s bo u n d to c of , i m i ted; ya I i di ty beca u so it can sa m p I e only a srn a N 
proportio iglish work; 

(b) like all as It Hsh, it suffers, despite riiarkcr^staridardisat]^^^^ 
frorri scrlrv . inconsistency, whicii probably causes some 
25 '\, nf cai < ; ' i\n marks a full band away frdrii those they 
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affecting the more heavily weighted assessment. Equal weijghting would also 
ensure teacher- and pUpil-mdtivatidn. If the school assessment covered a number 
of difTerent types of writing - say three from 'Expressive^ Factual, Persuasive, 
fictional Story, Dramatic Dialogue, Poetry' - the external test xould consist of 
a single topic to be dealt withjn a specified_styfc or with^a purpose ari r J 

audience in mind: this procedure would improve the reliability of the extenu*;! 
markirig but wouW n^^ 

would have to engage in several kinds of writing for the internal assessment. 
The topic and style specified in the external test could vary from year to year. 

An arrangement such as is suggested would improve the reliabitjty of the 
external assessment and provide a more satisfactory rank order upon v/hich the 
SCEEB examiners could fix band boundaries according to their judgment of 
standards. Any consistent departure by the schooJs in the internal assessment 
from SCEEB standards could be corrected by scaling iriternal against 
marks, wiiiie the combination of the two marks would reduce the effects of 
iriter-marker jiricorisisi^^^ would^ however, be ne^:epary toj^ yut^ome 
moderation of assignments set in the schools to ensure their validity: This might 
be CO m bmed wi t h Tri a I M a i"^ ing, or _effected by sc h d d Is send ing cd pi es d f 
assignments to SGEEB or to another appropriate body. 

The researchers believe UiaJ at present it would be^desira^ 
assessment of interpretatioji in the hands of the external examiners, because of 
t h e lac k of cp mj>arabj li ty^ be t we^ri i n terp rctat i on test s in d i ffe reii t ic h d dl s . Th e 
SCRE project has not, however, studied the assessment of interpretation as 
closely as that of writiiig^^ investigatidn df the 

validity and reliability of the traditional interpretation test would show that it, 
tdd, has itldre faults than is commonly realised. The whole quesftion of Jhe 
relative weightings of differerit elements in S4 English assessmeri 
recdnsideratidii as a result of such an investigation, perhaps so that greater 
weight was given to the response to |it_era_tu_reiLO 

of reading other than literature and interpretation passages acquired somCv 
significance. Schpols cpuld c^^ Pf testing reading: 

so long as it is necessary to produce a school rank order, however, it will be 
desirable thjU there be at least dne test cdmitldn to Jill classes, with a common 
standard of marking. 

The combined assessnient scheme suggested would, jn pnndplei allpW a re- 
duction in the size of the external examination, since the writing task could be 
shorter (at least when certa^^^ dLtask wer^ setiand it would be jJdssible 
to leave the assessment of literature wholly to the schools, if one was prepared 
IP rtcce^t the dMM^varitage that U intenial writing 

and literature against external writing and interpretation marks. Depending on 
success in develdping them and their acceptance by Engliish teacliers, multiple- 
choice tests of interpretation might be employed, which would reduce rnarking 
costs. The SCRE data has indicated that appropriately weighted combinations 
of writing and either traditional or rKultipJe-chpjc^^ 
fairly satisfactorily the range of skills also tested by the Folio Assessment: 

The practical problems of interna! Msessment wdU Id still apply to a cdmbined 
scheme, but jevould be more manageable without the need to set interpretation 
tests on Jhe O-grade pattern dr td prepare pupils fdr an external Hterature test. 
The researchers believe that the professional advantages to teachers in the ac- 
qUisitidri df kndwledge about, and practice in, asseissment, along with the im- 
provement in validity and reliability such a systerK wpuld prpba 
weigh 'the difficulties it would meet^ JsTevertheless^ it would be advisable to 
reduce the burden of assessment carried by teachers by requiring internal assess- 
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itient of ^bbUt teri j>ieces of work, rather than the fourteen or fifteen demanded 
by the SCRE project: 



4. Moderation . ^ ' 

sThe researcheris' views on moderation of iiiternai assejsrhent^r^imjhcit inlthe 
pxeeedjng discussiQn, Some mbderation of courses and assignments would be 
nec^essary and would probabJy be welcomed by the teachers as helpirig^ them 
to pl^ri their work better arid providing them with new ideas and suggestions. 

As t<:KmbdeTatiori o^^ the SCEEB examiners is the 

touchstone, and so long as it is important to effedt the kind pf discrimination 
among pupils which the existing examiii^ the mo$t effective way 

of applying the touchstone to a large number of internal assessments is probably 
^^1^1^?^ ^ "^f^^enoe test such as the extenwl writjnsan test 
mentioned in . above. Scaling of internal marks might, however, be more 
helpfully carried out against raw external marks, to ^^^a^ exaggeratipri of 

discreparicies jDri_r^ occurs if scaling is against external marks 

standardised in the SCEEB pattern: 

TJiere is a need for EnglLsh tea^^^ markers to be made 

aware that reliable fine discrimination is not possible between one pupil's *j3er- 
fonnance' in English^ ^^^^^ a: large mass of 'average' 

pupils. It would be preferable for markers and teachers to think of discrimination 
in terms of 5 bands rather than in terms of 14 ranges or 50 or 100 riiarks. 

5, Tendrsrs' prtqriries? ^ i_: l _ __ _ l 
TJie researchers' _(^^^^^ with the problems of discriminatoiy 
assessment did also lead tnem to consideration of the appropriateness in English 
of placing so much eriiphasis on this purpose of assessriient as opposed to others. 

Future development jS the assessment of English in S4 depends ultimately 
on value judgments abput^f^^^^ and about 

the degree of control ^ver curriculum teachers should have: The scheme put 
forward in the booklet oil assessment pubUshed by Jhe_Australian Council for 
Educational Research (ACER, 1974), quoted in 'internal Assessment or Ex- 
ternal Exam?' (Spencer, 1975), puts the matter neatly. 

' The article stated : 

'Methods <>f assessment can be placed on a co^^ in- 
creasirig levels of public trust and confidence in teachers, (b) decreasing 
concern for discrimination and competitiveness aiid inct^asinj; desire to 
haye aidescn^ rather than a statement of 

the quality of his performance relative to j>thers in his group. The nearer 
an examining bq^^^ the left of this continuum the more 

important is standardisation of any internal assessments; the nearer the 
!^oard is willinj to be to is for teachers 

to be trained in curriculum development and evaluation.' 

External exam at end of | Various types of moderated ■ Assessment without 
yea r or cou rse school assessment competition 



Increasing levels of public trust in teachers 



Nbrrri-based exanri results 

oil a certificate. 

Concern for discrimihatibh. 



Report of pupil's achievements. 
Concern for diagnosis and 
feedback to pupil and teacher. 
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Rcaiisatloii of ilic inadequacy of discriminutdfy assessnieiit in Engiiish coiiid 
have fruitful results for other sorts of assessment. The general reappraisal of 
assessment in S4 occasioned by the proposaLs of the Dunning Committee 
provides a context in which^a shirt in emp^^^ assessment of English 

could occur: The in-service assessment tniining recommended by that Committee 
iHpuId concentrate fi rst ori assessment design . Cpurse-pUinri irig takesji pri ntary 
place iathe process, and the assessment is designed to match the purposes with 
which Jhc teacher taught and the pupils learned. English departments need to 
develop consciousness of all the possible elements of the English curriculum 
suitable for their particular pupils and then, in planning a term's or a year's - 
course, to make clear choices among them so that they can specify (yet without 
inflexibility) the purposes for which they and their pupils wi!i work in the im- 
mediate future. iWhen the assessment is cjos^ related to particulAr purposes in 
this way it is possible to obtain a more specific description of achievements in 
English than if the principal conceni is to dis^ 

ment of this sort can result in the diagnosis of strengths and weaknesses or in 
the no t ing b f a ttempt s by t he p Up i 1 at U sages which show that h i s I irigU i st i c 
awareness is developing - outcomes much more closely related to the business 
of teach ing and learning than are grades achieved in a discriminatory examination. 

Efforts to seek comparability of standards should not be abandoned, but the 
state of English testing and teaching rnight be the better for a more general 
awareness that it is n&t in practice attainable, even in a good external examina- 
tion which takes careful measures tp e^^ as_ much fairness, to all .candidates 
as possible: If this knowledge permeated the consciousness of English teachers 
at t he _sam^^pje_ as t H cy were gi yen a s ign ificarit weigh t of jiM^ssm en t to ca rr^^ 
out thejjjpdves, the powerful influence of the external examination on what is 
taug^ht in S 4 \vb u 1 d be w eakcncd , m ore varied co u rses co u I d t h cil be de vc I oped 
in more schools, and the quality of those courses could be more immediately 
and more effectively evaluated by the teachers who designed and taught them. 
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Appendix 1 

Questionnaires tJsed^in Survey of English Teachers' Views, 1975 
(see Gtiapter I) 

Appendix 1(a)': Free R^pohse Qnestibniialre 

Thris is not included in this publication but is available as a separate item, on 
request, from the SCE Examination Board: 
^N.B. The ^Objectives of the Orcltnaiy grade assessment of English' which the 
teachere commented on can 'be seen in a very slightly different version 
in Appendix 3, page 111. 



Appendix i(b): Fixed Response Questionnaire (with results) 



Name of school . . > 

Please tick the relevant boxes: 



Pdsitidn in English Department: 



1. 


Principal Teacher 


( 


) 


89 


)7% 


2. 


Assistant Principal Teacher 


. ( 


) 


79 


15i% 


3. 


AssistPjt. 


( 


) 


355 


67-9% 










523 


100% 


!o. 


of years teaching English : 










4. 


0-2 


( 


) 


111 


21-2% 


5. 


3-5 


( 


) 


126 


,241% 


6. 


6-8 


{ 


) 


65 


■l2-4% 


7. 


9 - 15 


( 


).. 


102 


19-5% 


8. 


16 or more 


( 


) 


119 


22-8% 










523 


100% 



NOTES 

1 . P^fi^i^lofi of Vhjectives. Speak ing, read ing, wri ting, and the assessmen t of 
success in these activities, are complex^ functions involving sets of skills, 
personal characteristics and experience. Most pieces bf writing, for instance, 
will show that pupils have achieved objectives in several of the categories 
set out in this questionnaire. The classification used will, it is hoped, clarify 
what •performance^ in English inyblvesi^ but it would not be desirable, or 
possible, to assess the achievement jof each objective separately. Farther, 
the jist is not ebrriprehe^ 

such as the stage of intellectual and emotional development a pupil has 
reached, cannot be accurately measured with existing testing tooJs^ T^ie^^^ 
are also^ even at 0-grade standard, qualities recognised as intrinsic to good 
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communiciition - eg:, bicadih and aptness of vocabulary and variety and 
approj5riateness__o Sentence striiclujx: -^^ ^vll'cll specilic cH 
be formuiated in advance, the purpose of the questionnaire is therefore 
to obtaiii an overall impression of the collective state of niind of English 
teachers rehit!ve to objectives which can be spec! Hcd with some prcci^idn, 
but the research teani remains a\\arc that not all the qualities of *gooci 
English' are definable in advance. 

2. Cfc-jr/y/ra//o//. Other kinds of classihcatipn of types of wntirig ajid oliiectives 
are possible. (The one used here is. derived directly from Jamci Britton's 
*What's the UscTiij^. LmWuigle m MH<r'<^JWfh.V3M Tpj* th>: Open Uni- 
versity by Routlcdge ancl Kegan Paul:) the research team will welcome 
cbmriierits from an 

type of classification: It is, however, essential in order to maintain com- 
parability ariibrig all the respondents that the questionnaire be fjllcd in us 
it stands. 

5. Standard \l will help throughout the questiphnaire to keep in mihcl the 
O-grade standard, which is defined ^is being *such that a pupil who is. at the 
lower end of the top 30 per cent of an age group should^ with satisnictbry 
teaching and .adequate efforts on his part, have a reasonable prospect of 
securinjg passes U.e.^ Grade Cl^ v a M ^iye^st /7:r<^e subjects in the fburth 
year'. ('Report of the Working Party oji the Curriculum of the Senior 
Secondary School', _HMSp^^ |959 ) there are no figures avail- 

able to indicate the proportion of an age group expected to obtain Grade C 
ih Eiiglish ajorie,^ but a reasonable estimate is that, in the conditions men- 
tioned above, a pupii at the lo wer emf of tlw top SO per cent of an age group 
is likely to perform in the fourth year at about the Grade C/D horilerUne 
in English alone. 

4. ^Jargon', technical language from various ficjds of Jtudy has been erii- 
plbyed in the questionnaire for convenience and in pursuit of precision: 
it is not implied that it should be used in the classroom. 



Part One: Objectives 

WRfTING 

Three main headings are proposed: 

\a) -Trajisaetioriar writing: tb dircct, question, get things done, participate 
in society ... 

{h) 'E?Cpressive' writing: to exchange or reveal feelings and opinions, convey 

attitudes, reveal personality . . . 
(cj 'Poetic' writing: to cpnstruct ljh^mstic aru^ - 

as a means of trying to explore and master the complexities of reality. 

Please tick the appropriate boxes to indicate which objectives under these 
general headings are, in your opinion, the niTnimuhi a pupil should be capable 
of achieving to obtain (i) Grade G, (1!) Grade A in an assessment of O-grade 
English. (Some may be thought more difficult thtui the minimum requirements 
for Grade A:) ^ 
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TRANSACTIONAL' WRITING 
Pupils should be able tp: 



4 

9. write instructions Tor a simple task (recipe, care of 
spoHs ecjuiptneht . . .) 


Grade C 


G r adef A 


No. hot 
choosing 
Item 


498- 
95-2% 


la 


15^ 

2-9 y 


10. fdnnulate rules for a cliib, social evening, or some 
sUiiatioii in vOliicH firbiib behaviour hiust be 
controlled 


264 
50*5% 


493- 
36-9 % 


66^ 

12-6 V 


1 1 ~. report events factually 


488 
93-3% 


24 

4-6% 


11 

21% 


12: write notes on events as they happen 


312 
• 59-7% 


141 

27-0% 


70 

13-4% 


13. write a report presenting information logically 


255_ _ 
48-8% 


235 - 
44-9% 


33 

6'3% 


14. order and paragraph ideas 


439 
83-9% 


69 

13-2% 


15 _ 
2-9% 


15. write an argument based on facts for or against a 
point of view 


26i 

49-9% 


245 
46-8% 


17 

3-3% 


16. present evidence and draw conclusions from it 


131 

25-0% 


341 

65-2% 


51 

9-8% 


1 7. give arguments for and against a point of view and 
draw rational conclusions 


18^% 


348 
66-5% 


80 

15^3% . 


18. illustrate a ^neral idea with a particular exarriisie 


263^ 
50-3% 


213^ 
40-7% 


47 

90% 


19 Hpvplnri hri iHftfl illit^trattnir niintiriff aHHiicihc 

17. Mil IVJWvl, 1 1 i Ud&l A VI IIK, ^UVlilI£*1 ^uuuWWiii^ 

evidence in siippdrt ^ 


62 

11-9% 


354 
67-7% 


107- - 
20-5% 


20. write ' a logical, theoretical argument, given 
premisses 


22 

4-2% 


302 
57-7% 


199 
38-0% 


21: write advertising copy (c.g: for school events : . :) 


350 _ 
669% 


124 

23-7% 


49_ 
9-4% 


22. write, a persuasive. argument using emotive lan- 
guage to win support 


56 

107% 


360 
68-8% 


107 • 
20-5% 


23. write a speech for or against a motion 


262 

5b 1 % 


22i _ 
430%, 


36 

6-9% 


24. summarise factual information 


445 
85-1 % 


67 

12'8% 


11 

2-1% 


25. summarise an argument, maintaining the logical 
thread 


171 

32-7% 


291 


61 

n-7% _ 


26. write personal letters of various kinds 


497 
95<1% 


16 

3-1% 


10 

1^9% 


27. write formal letters to achieve practical ehd.s 


462 
88-3% 


49 

9-4% 


12 

2-3?/ 




TXPRESSiVE' WRiTiNG ^ 
Pupils should be able to: 





Grade C 


Crncfe A 


No. not 
chmstng ' 
item 


28. describe their own feeling about and, as stimulated 
by. events they have experienced 


479 


35 _ 
6-7 V 


9 


29. deRne their attitudes to aspects of society which 
impinge upon them (e.g. authority, other groups, 
other rac^) 


279 
53-3% 


205 
39-2% 


39__ 
7-5% 


30. state bpiiiibhs on controversial topics 


309 
59 1% 


177 

33-8% 


37 

7 1% 


3 1 . show awarenciss of the causes of their feelings and 
attiludes 


104 
19-9% 


296 
56-6% 


123 


32. convey their interest in various topics by writing 
knowledgeably about them 


^3^ 
77-1% 


105 

20^1% 


15 

2-9% 


33: write fantasy stories (invented situations) 


336 - 
64-2% 


24-7% 


58 - 
111% 


34. write imaginatively (i.c: with originality) or criti- 
cally expressing a personal view given an artistic 
or natiQ^ .stimaliB (tilin; poem; music; picture; 
physical experience . . .) 


177 
33-8% 


283^ 
541% 


63^ 

120% 




1Q4 102 



TDETIC WRITING 
Pupils should be able to: 



35. wrile fortes with sornc exposition of the causes 
and effects of events 


Grade C 


'Grade A 


dtussing 
item 


264 
50-5% 


' 196 _ 
37-5% 


63 _ 
120% 


36: ofpnisc events in a story to create a climax 


278 _ 
53-2% 


215 
41-1% 


30 

5-7% 


37: create suspense in a story 


M72 
32-9% 


277 
53-0% 


74 

14-1%; 


38. write Jilojric&Jn which the feelings of more than 
one character are revealed 


152 
_ 29-1% 


283 

54-l% _ 


88 

16-8% 


39. w_ritc stories with genuine interplay between events 
and characters 


63 

120% 


299 
57-2% 


161 

33-8% 


40. reveal character in a story through dialogue 


118 
22^% 


295 
56-4% 


na 

210% 


41. dramatise (not just describe) events in a story 


100 
19-1% 


273^^- 
51-2% 


15a 

28-7% 


42. write a story with a rhoral or message 


228- 
43-6% 


18? 


106 
20-3% 


43. write adopting the poirrt-of view of someone with 
feelings and attitudes different from their own 


110 
210% 


277 
53 0% 


136 
260% 


44. write with different degrees of fdrniality to 3ait 
circurristahces - 


174 
33-3% 


259 
49-5% 


90 

17-2% 


45. write in simple poetic forms (e;g. ballad, haiku, 
free Verse . : :) 


149 

28-5% 


182^ _ 
34-8% 


192 
36-7% 


46: write in stricter poetic forms (e.g. blank verse, 
sonnet) 


2 

0-4% 


156 
29-8% 


365 

69-8 %,__ 


47. exploitidn prose or verse) tlhe emotive overtones 
* of words 


23 ^ 
4-4% 


224 

42-8 %_ 


276 
52-8% 


48. use language figuratively 

- - 


149 
• 28-5% 


287 
54-9% 


87 

16-6% 


49. exploit the sound and rhythms of language to 
make a stronger irnpression on the reader 


43 ! 224^ 
8-2%' 42-8% 


25& 
48-9% 



Knowledge of appf opf iate vocabulary and x)f a variety of sentence stru^ures 
is inherent in the objectives listed abov^^^ 

is employed would be assessed in judging the standard of any piece of writing 
by comparing it with other wbr^^ would a! sd be taken 

into account. While it is not practical to specify in detail minimum criteria in 
grammar and_speliing, some iridicatib^^ views about formal correct- 

ness may be obtained from the following section of the questionnaii^. 
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. Please^ iLck the apprbpriate boxes to ihdicate/your view of thp minimum re- 
quirements at each grade. Pupils 1n their norjhal writing should be able to: 



- -- 




Grade A 


No* not 
chodsfng 
ttem 


50. Use full stops correctly in nearly every instance 


499 
95-4% 


19 

3-6% 


5 

10% 


51. Use the question mark correctly when required 


501 _ 
95-8% 


14 

2-7% 


8 

1-5% 


52: use quotation marks for Direct Speech 


496 
94-8% 


21 _ 
40% 


6 

M% 


53. use the comma correctly to rhark off subordinate 
clauses and parenthetical or appbsitiohal phrases 


284 
. 54-3% 


215 
411% 


24 _ 
4-6% 


54. use th& contnia correctly in combination with 
quotation marks 


391 

74-8%^ 


10] 

_19-3% 


31 

5-9% 


55. use the exclamation mark correctly 


755% . 


104 
19'9% 


25 

4-8% 


56. use the colon correctly 


448^ 

22-6% : 


288 
55^1% 


117 

22-4% 


57. use the semicolon correctly 


22-6% 


298 
57-0% 


407 
20-5% 


58. use the apostrophe correctly 


454 . 
86-8% 


5a 

1M% 


11 

2-1% 


59. use quotation marks correctly for slang, foreign 
phrases ... 


268 
51-2% 


197 

37-7% 


5a 

11-1% o 


60. " use parentheses correctly 


148 _ 
28-3% 


277 
530% 


98 

18-7% 


61 : Use the dash correctly 

■ 


125 

23-9% 


285 
54-5% 


113 _ 
21-6% 


62: Consistently acxUrat& spellih should be a prc- 
requis^e for the award of 


174 

_33-3% 


263 
50-3% 


86 

16-4% 


63. Pupils should regularly show evidence of skill 
witli tenses, subordinate clauses, ribuh phrases, 
appropriateiiinking words and other aspects of 
grammar iiontributing to the complexity and 
variety of sentence structures to obtain . . . 


107 

20-5% 1 

j 


343 
65-6% 

1 

1 


73 

14-d% 
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READING / . 

Please indicate as/oefore, your view ofthe minimum requirements at each grade 
by ticking the appropriate boxes. 

As regaids tneir reading, in or out of school, pupils should be able to: 





Grade C 


Grade A 


No^ not 
choosing 
item 


64. rei^oduce the gist of^ narrative 


507 
96-9% 


8 

1-5% 


8__ 
1-5% 


65. state their own emptipnal reaction to the content 
of what, they have read 


.428 
81-8% 


79 

151% 


16 

31% 


66. recognise and state the writer's attitude to his 
subject, and some of his ideas 


28(5 


222 


15 

_2-9% 


67. state the tohe/feelihg of what they have read 


168 

32-1% 


306 
58'5% 


49 

9-4% 


68. state their erribtiohal^^nd/br moral reaction to the 
writer's treatment of His siibject 


200 
38-2% 


2ia 

47-8% 


73^ 

14^0% 


69. recognise and comment oh the accuracy with 
which literature reflects hiimah characteristics,, 
relationships and problems 


87- 

16-6% 


278 - 
53-2% 


158 
30-2% 


70, recognise and refer Id the writer's techniques (e.g: 
devices for creating suspense,^ revealing character, 
or stmctaraiiy relating parts to the whole, his use 
of figurjtiive ianguage^ or other means he uses to 
creatCLan emotiq^al and/or moral impression on 
the reader) 


84 

16-1 % 

j 
1 


345 - 
66-0% 


94 

18-0% 


71. give_rcasoris for enjoying what they have read in 
terms of the writer's skill 


196. _ 1 
37-5% i 


268 
51-2% 


59 _ 
11-3% 


72. give reasons far the judgemeni that one piece of 
writing is superior to another 


122 1 278 123 
23 3?<; j 53-2% 1 23-5% 


73. locate specific infonriatipn ifj a book using Con- 
tents. Index, Chapter headings etc. 


'424 1 58 41 
81-1% : L .7-8% 


74. *skirn' read to gc t a general impression of a passage 


182 -1 242 1 
34-8% 1 46-3% 1 

\ — ^- 1 
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eeMPREHENSION 

When cari7ing out *cIose reading' of a piece of writing, pupils should be able: 





Gr a c/e C 


Gra(/e A 


cho.ashig 
item 


75. to recognise or recall information explicitly stated 


492 

94;1.% : 


19 1 12 
3-6% i 2-3% 


76. paraphrase (^r translate such infbrniatibn 


426 
81-5% 


8! 1 16 
15-5% 3-1% 


77. .-fhake ihrerchces based ori ihfbrrhation explicitly 
slated and on their own experience 


266 
50-9% 


221 1 36 
42'3% i 6-9% 


78. recognise or slate the main points presented, as 
opposed to digressive, illustrative or secondary 
ones 


389 
74-4% 


42a ! 14 
22-9% ' 2-7% 


79. reorganise and paraphrase (i.e. summarise the 
main ideas presented 


404 
77-2% 


105 
20-1% 


44 - 
2-7% 


80. make a judgment based on their owm knowledge, 
experience and values about the validity of ideas 
in a text 


128 

24-5% 


269_ 
51-4% 


241% 


81. distinguish factual writing from opinion or per- j 
suasion * | 


336 
64-2% 


164 
31-4% 


23_ 
4-4% 


82. state Jhe cffecL_on_raeanin& Of tODp in con text of I 
the use of different registers of language ' 


90: ' 

17-2% ; 


327 

62-5% j 


106 

20-3% 


83. «iate the effects in context of particular sentence [ 84 1 319 | 120 
structures I6-I% | ei-O!*^ | 22-9% 


84. state the effects in context of particular usages of 1 35 ' 289 ' 99 
grammar ajtd punctuation _ : 25-8 % _L 55:31{,_ j 8-9% 


85. explain the implications, overtones^ cohhbtatibhs ! 83 

in cbntext of language used figuratively 45-9% \ 


323 i 117 
^ 61^% 22-4% 


86. stale the general idea represented by a particular i 268^ j 
ificideht, detail or irnage | 51*2% ' 


201 1 
38-4% ! 

. . .._ 1 


54 

10-3% 
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Pupils* knowledge of janguage ought to include the foiiowing concepts: even 
though they, ma^ hot be able tp: use the. tenn^^ correctly, they should be 

able to explain the function of each in context. 



87. Noun 


Grade C 


Gr a<^ A 


No* not 
choosing 
Item 


502 
960% 


2-3% 


9^ 

_ ._l--7% 


88. Verb 


503^ 

96-2% ' 


n 

2-1% 


9^ 

1-7% 


89. Adjective 


SOZ 
960% 


12 

2-3% 


9: 

1-7% 


90. Adverb 


476 
91-0% 


29 

; 5-5% 


18 

.3-9% 


91. - Pronoun 


466 
89-1^, 


36 • 
6-9% 


21 

40% 


92. Sentence 


495": 
94-6% 


3-4% 


id _ 

1-9% 


93. Clause 


323_ _ 
61-8% 


131 
2-5% 


69 

13-2% 


94. Adverbial clause 


123 

23-5% 


225 
43:0% 


175 

33-5% 


95. Adjectival clause 


124 

23-7% 


226 
43-2% 


173 

33-1% 


96. Noun clause 


106 

20-3% 


228 
43-6% 


189, 
36-1% 


97. Phrase * 


418^ 
79-9% 


6a 

13-2% 


36^ 
6-9% 


98. Subject - 


445- - 
79-5% 


52 

9-9% 


55: 

10-5% 


99. Object/Complemeht 


334 
63-9% 


96 

18-9% 


93 

17-8% 


100: Singular/Plural ' f 

■ V 


477 
9t-2% 


17 

3-3% 


29_ 

5-5% 


tot: Tenses 


470 
89-9% 


33 : 
.6-3% 


20 

3-8% 


102. Past and Present Participles 


236 
45-1% 


181 

34-6% 


106 

20:3% 


103. Finite/Infinite moods 


108 

20-7% 


212 
40-5% 


,203 
38-8% 


104. Subjunctive mood 


48 

9-2% 


213 
40'7% 


262 

50-1 % 


105. Active and Passive voices 

! 


186 ! I9a 
35-6% 38-0% 


438 
26-4% 
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ORAL ENGLISH , / 

Detailed objectives have not been worked: QuH'CH^his aspect of gnjgiish work, 
but it will help tq know teachers' vieVfs about the need-fbrdovelbpmerit of it. 

















Agree 


Disagree 


response 


106. 


There is a heed tor Oral English courses to be 
planned 


.177 
72-1% 


109 
20-8% 


37 


107. 


If it is to be assessed, it is better to assess oral 
perfdrmaricc completely ihrormally, without a test 


316 
60-4% 


162 i 

31-p^/o i 


45 

8-6% 










108: 


There is ho heed to try t(> assess oral English 


123 

23-5% 

I 


324^ ■ 1 
62-0% I 


76 

14-5% 











Part Two: Methods of Assessment 

Please tick one box for each item to incf jcate your opinion of the usefulness of 
the existing O-grade English exatitinatit n for the follo\Virig piiipbses. 



109. to give to cmpldyers and 
thbsc^espbr4sible for Fur- - 
ther Education uscrul in- 
formatibh for their selec- 
tion processes 


1 Of some 
Effective j use 


Soiyety 
effective 


Probably^ .^o. 
useless \ response 


90 345 
17-2% 66-0% 


65 

12-4% 


22 * 1 
4-2% 0-?.% 


110: to provide informatibh for' i 49 259 133 
the teacher abbiit the qual- | 9-4 % 49-5 % 25-4 % 
ity of the course he has 
taught ; 


74 8 
14-i% kSX 


111. to provide pupils with in- ! 
fbrroation-Hboat their pro- ; 
gress and achievements 

1 


80 i 318 94^ : 
15-3% i 60-8% 18-0% 


22 9 
4-2% , 1-7% 


1 12. Lo jTioiivatc pupils to work I 
harder 

1 


119 ' 268 ' 92 
22-8% j 51-2% : 17-6% , 


33 ■ 11 
6-3% 2-1% 

1 



Please tick the relevant box to provide information about the following: 





! 

j res 


No ' 


No 
response 


113. This_schQol is using a system of continaous assess^ 
ment of Englkh^ or thciissessraent of a folio of 
work, in some or all of forms S1tS3 


: 305 
! 58-3% 


m 

36-9% 


25 

4-8% 


1 1 4. "Diisschoi)! is using^ jsystem of jusn tinuons assess? 
ment of En^ish^ or the assessment of a folio of 
work, in some or all of forms S4-S6 


113 
21-6% 


359 
68-6% 


51 

9-8% 


1 1 5. I would welcpme the introduction of someXorm of 
suitably moderated schooi-based assessment of 
English at O-grade 


266 _ 
50-9% 


219 _ 
41-9% 


38 

7-2% 



Mo 



Part Three: Cdmitibrits 



Please feel free to comment oil any aspect of this questionnaire: the research 
team will welcome any criticisms, suggestions or statements of opinion. 



^PP^?^^ Assisssmcnt Used in Drawing Up' 

Order of Merit Lists for Candidates Sitting the 1975 English O-grade Examination 

This is not included in this pubiication but is avaiiabie as a separate item, on 
request, from the SCE Examination Board. 
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Appendix 2 

Consultants \vho Contributed to tt e Making of the Criterion Test 

Mr A. D. Biithlay, Senior Lecturer in Eni'Jish, Aberdeen College of Education 
Mr R. S. Fyall, The High School, Dundee 
Mrs E. Grainger, Glasgow 

Mr I. G. Mathleson, Principal Teacher of English, Whitfield High School, Dundee 

Mr D. Mejizies, Adviser in English, Strathclyde Region, Lanark Dfvision 

Sister B. Mbljoy^ formerly Assistant Principal Teacher of Erigh'sh^ Lawside 
Academy, Dundee ' 

Mr M. J. Morris, Senior Lecturer in Engiish^ Bell Coilege of Furtjier Education. 
Hamilton 

Mr J. P. O'Neill, Senior Lecturer in English, Jordanhill College of Education 
Mrs B, Ramsay, Edinburgh 

Mr t. A. Sillars, Depute Head Teacher, Auchenliarvie Academy, Stevenstdn, 
Ayrshire (now Headteacher, Thomas Muir High School, Bishopbriggs) 

A pa rt ic u I a rl y sign i fican t con t ri b u ti ori to Paper I I wa s mad q by_ Mr J. Ihglis^ 
formerly Principal Lecturer in English, Jordanhill College of Education. 

Thanks are expressed also to the many headteachers and teachers who aliowed 
the researchers to pre-test parts *)f the Criterion Test in their schools. 
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Appendix 3 

Stag^ in tiie Develdpiherit of a Criteribii Test ^ 

I- Preparatory Work: Objectives, List of Skills! Suitable Materials and VArious 
Constraints 



(ffj Definitioh crf Terminojagy 

The remitfor the researchers referfed to 'objectives, as stated by the SCEEB, 
of the Ordin^i^ grade, assessment of English as at present constitj^ed', and also 
to 'other objectives; if any; of the study of English in secondary schools which 
are npt currentjy as^^^ refer^ncei in the description of the 

proposed 'comprehensive testmg procedure'; was to 'all aspects of the perfisr- 
rharice of pupils studying Enghili^ in S^^^ preparatory for presentation 

for O-grades'. Despite some ambiguity, it seems fairly clear that teaching ^)r 
learning' ^^y^ctjveis pl>jectiveSj such^^a^^ 

instance, discrimination among various groups of pupils. The 'objectives' were, 
however, 4inked to the O-gradt examinatldn, so that some cdrrfusidn with the 
criteria of success ih>that test was possible. 

The use of the term 'pbyec^^^ rernit created conceptual difficulties 

which were actually only^encountered and removed as work progressed, but 
vvhich, for the sake of clarity, are discussed here. 

^Were teac h^rs ' real or stated teae h i ng b bjec ti ves un der study ? How were 
' objjectives to be identified? Investigation of teaching and learning objectives as 
cy^^^"ced by and pUpils' behaviour task which 

could have been effectively carried out within the time limits of the project 
without^ several more researchers. Should there be an attempt to apply to English 
some version of Bloom -s Taxonomy of Educational Objectives^ Apart from the 
fact that many English teachers would decry such an exercise as Hkely to dver- 
simpiify theisubject: a^^ over-explicttiiess, inhibit teachers' curricular 

adv^nturousness; there were theoretical objections to an 'objectives' approach 
to Eri|;] i s hj ^ari sing^ f rpni_ Lawrence iSt eh h bu se's cbnsiderati on of th e nat lir e of 
work in social and aesthetic subjects.* His argument implies that in teaching 
English, one does "P^ first "define aims and;"0tgect^ cbJ1t^Mlt 
and methodology to attain them, rather, the coT/rewr of the work is central: 
English consists df the expldratidn in language df *cdntent', .whether it be texts, 
experience, reactions to stimuli, emotions . , . and, far from progressing towards 
predictable ends, one hopes that some of the results will be surprising td pupij 
and: teac her. The _E ngj ishj teiac h er^S: busi ness ] n Sten ho u se' s v ie w, is to eh d dse 
content which has value and is appropriate for his pupils and to present it for 
explbratioh, not with defined b^eetiy^s^ in m but with 'purposes' flexible 
enough to adapt to unexpected developments: 

The author, who conducted the early part of the work alone, interpreted the 
fuilctidii df the prdject as mdre pragmatic than a .theoretical cbhsideratibh of 
the nature of English as a subject. He was in sympathy with Stenhouse's thesis 
and took the view that any exhaustive taxdnomy df dbjectives, after BIddm, 
wbiildi in any case, be based on observation and description of skills actually 
exhibited by people using English: The instruction of the remit to 'note the 
Pjy^ctjyes . . . of the Ordinary Grade' and to 'list . . . other objectives' was 
therefore taken to mean something which did seem within the possibilities 
defined by the size and time limits of the prc^^^^ 

mentsfor which credit is given to pupils by the 6-grade examiners and by teachers 
»Steiihourcli970^7I) 
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in assessing aiiy additibiial work done fn sclibdl. The emphasis fell oil *what 
you give marks or credit for', and broad teaching aims were inferred from the 
cri te ria of success^ The di fficujty of work ing in ariy^ other way is, incidentally, 
one reason why assessment systems so powerfully affect English syllabuses: 

The terrri 'objectives\ was retained for the t\vo questionnaires on Ehglish 
achievements; it was not explicitly defmed, but was obviously not being used 
in a: strict bchayiburist scns^^^^ project the expireSsion 

'List of Skills' seemed more accurate and was more acceptiible to most English 
teachers, smce it docs riot carry the scierit i ficibr dctcrniiri i stic overtones some- 
times associated with Bloom's use of the word 'objectives'. 

(h) Strategy ami Practical Limiiot ions 

The first plan was to produce^a 'comprehensive testing procedure', or Cfiteridn 
Test, by creating test assignmcnti. and questions derived directly from, and 
covering all of, a full list of the English skills which attract credit in the O-grade 
or in school assessments. The list was to be compiled after consideration of the 
stated aims of tlic SCEEB examination, analysis of both questions actually set 
arid rriarkirig criteria afjplied, and cbrisultatibri with exarriiricrs and teachers. 

Essentially, this was, indeed, the way by wh|ch the liiial Version of the Criterion 
Test was arrived at, but some snags were met en route. There were three sorts 
of problem. First, it became dear that a^ Criterion Test covering the whole of 
a cornprehcnsive list of skills would last about riinetecri hburs. Secondly, certain 
aspects of English work, included in the description of all posslbie S4 worjc, 
were: not Jn fact^_ bemj uridertakcri b^ any bf the schogls whjch by this time 
had agreed to participate in the experimental internal assessment scheme. There 
was ri b point i ride vis irig a test of p^ipi 1 s' ab i IRy to recogn ise slan ted p ress 
reports, for instance, if none of the pupiJs concerned had been taught the skill. 
Further, because the schools were still taking the O-gfudc examirtatibn, teachers 
were concerned that pupils should be familiar with its format. The question of 
the assessment of oral English in the Criterion Test \vas an associated problem. 
It also would have entailed unfamiliar asscssmerit prbcedures. In the events after 
the r^isearchers had made some attempts to develop and pre-test tests of oral 
English,:the Steeriri^ Cbmmittce of the j3rojecti Jecided tMt there shbuld be no' 
oral assessment in the Criterion Test because of the special difficulties of stan- 
dardising i t_ and because it was 'stray irig too far' frojii the purpbses of the 
present O-grade examination. The third difBculty was in finding test material 
writirig stimuli, passages for iritcrpretatidri, literature questions - which covered 
the list of skills, were not too unfamiliar to tke pupils, and were such that they 
allowed pupils across a wide range of ability to show what they could do Jn 
English, Material chosen because it seemed to facilitate the testing of listed 
skills often met objections from the consultants on the grounds that it failed 
to meet oneipf the oter rcctulrem^ good examinatibri j^u^ 

in the case fef an interpretation passage, it lacked intrinsinc vakie as a piece' of 
w ri tin g, or wo ul d be no rii o re t h an a jneari i riglcss exerc i sc to the can d i date , in' 
the case of a writing assignment. Pre-testing of intended Criterion Test elements 
also led to the FCjectidri of several because they proved too easy, too hard, or 
too incomprehensible to the pupils. 

Tiie Cri teri on Test as event ually set was, thercfo re, the bu tcqme bf_ cbm- 
promises: It was decided that two days was as much as could reasonably be 
asked of schools for the admiriistratibri bf this test, §b it was restricted to about 
id hours' duration. To allay the anxiety of teachers about examination practice 
for O-grade,. the first two papers of the Criterion. Test were -to take thc^same; 
form as the O-grade examination. The assignments actually employed in the 
test were cho.sen because, of all those pre-tested and discussed with the con- 
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su Hants, they best met the criteria of interest^rid apprbp^ 

tioii over a wide ability Tange and at the same time seemed to lest satisfactorily 

those skills selected for iriclusioh in the ho longer fuUy comprehensive Criterion 

test; 

(c) Lists ofSkiiis 

' As c^ be inferred from the account of constraints on the size and form of the 
Criterion Test in (6) above, not all of the tptal list of sRills produced__by th^ 
researchers, libr all the insights gained in the process of compiling the list; were 
actually employed in the making of the: test._ The tojal liit and d^^^ 
the Various stages through:which it passed may, however, be valuable for other 
purposes. The anaJysjsjof.Enjlish cam^ 

matic one^d did not pretend to bQ exhaustive^ to draw fine distinctions between 
degrees lof sophistica^^^ of to lead to a 

statement of the nature and content of English as a subject basejd on any logical, 
sociological or psychological principles of organisation. It might, though, be 
regarded as having provided for some asp^ts of English >york -'check lists'^ 
which may be helpful in developing assessment for diagnosis of strengths and 
weaknesses, or for producing /profile' descriptions of pupjls/^achj 
English. Perhaps it is only through therefinement of such pragmatic descriptions 
of Englisii skills that one max succeed ih_prgd^^ 

of English as a discipline, or provide more precisely worded criteria for assess^pnt. 

The process of defining 0-grade English skiiis began with a consideration of 
the 'syllabus' of O-grade work laid dbSvn in the SCEEB's Condittons and Ar- 
rangBvients, i975\ This *syliabus' statement had not varied for some years prior 
to_[975. It was rioted that the statement was of a general nature and open to 
different interpretations. To obtain ^ list of more detailed criteria an^^ 
study was riiade of the questions set in the O-grade Englislipapers from 1969-74, 
and of the Instructions to Markers: for 1973 /and_ J_97?L TS^latter varied from 
year to year only insofar as wasjiecessary to meet the demands of particular 
assignments set: as faras Paper t^Compojitiori ^ a was concerned, 

the basic criteria of assessment had remained the same since the introduction 
' of the <)-iraie_exaMn^^ study was completed, the 

Principal Examiner and one:of his senior colleagues were consulted. They con- 
firm€d_t_hat the set of criteria and aimis in 'Objectives of the Ordinary Orade 
assessment of English as at preseiit constituted' was a fijII stAtenLen^^ 
takeri iritb account by the O-grade examiners. The list-was then sent to a sample 
of teachers for their conoiments, arid they, tob^ were gerierally satisfied that it 
fairly arid comprehensively described O-grade achievements. (The teachers' cbm- 
ments are reported more fully in Chapter I.) 

Under each heading, Cdmpositjori, Readirig^ Iriterpretetidri, the opening 
statement is a summary of the purposes of the Paper as set out in Conditions 
ond Ar^rangetm 1975; 'Criteria' were derived from the analysis of questions 
and marking instructions and from the oral comments of Jhe examines; 
Teachirig Airiis' were extrapolated from the criteria by the author. 

First List of Skiifs . 

ObjecHves of the Ordinary gca^^^ssessment of English 
as at present constituted - 

COMPOSmON ' 

To test ability to write Interesting and coherent cbntihtibus prose. 



The criteria for Jissessing_this_ ahiiity_are stated or implied in the SCEEB's /Guidance for 
Teachers on the Marking jjCJEn&lish Essays, Ordinary and Higher Grade'. While the ernphasis 
is on the total impression made by^'the sets: of inextricably linked skills which produce the 
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writing, the fdllbwihg points appear lb be considered by t'lc examiners: 

i. Relevance, quaJity^ depth and quantity of ideas and observations. 
2: Degree to which the *persbnality' and Torccfulness' of the writer are revealed in the 
composition. 

3. I^gree of sincere interest in the subject. 

4^ ^^8''^ °f orij^nality, Treshness' of approach to it. 

5. Degree to which flow of ideas is unimpeded by faulty punctuation, grammar and 
spelling. 

^6. Aptness and breadth of vocabulary: 

7' Pr^^^ring and paragraphing of ideas, including appropriate use of iDustration. 

8. Varietyjof sentence stractare, and suitability of choice of structure to the expression of 
ideas and feeling. 

9. Apprbpriateness of the register of language used to its context. 

Certain question choices seem to require also: 

jp. Ability to write in a specialised style (letter, newspaper article, report . . .) 

1 1: The presence of a greater interpretative arid imaginative pbWer than the other tbpics 

(Questions using poetry qaotations as stimuli.) 

Teaching Aims 

The teaching aims which these criteria imply are: 

1 . That_thei:aadidate_sh0uldhave-knQwiedge of varioas aspects of life. Including his own 
and others* attitudes and feelings and their causes. 

2. That his knbwledge should have grown out of genuine interest, or personal experience, 
and but bf maturity. __ _ 

3. Ijiat the candidate should possess the con^dence to express this knowledge: 

4. That he ^hbuld have eribiigh experience of the uses of language to have acquired the 
means bf brdeiing it and cbmmuiiicatirig it fully in writing. 

READING ' 

Xp test how *^ej 1 informed* candidate are about their reading,, both school work^and private _^ 
reading. |nj>petry Questions, to twt how far the candidate appreciates the poet*s presentation "^-^ 
of the subject, as Well as the content of the poem. 

Crjteria 

Consideration of the qaestions set in years 1968-74 reveals that pupils are expected: 
\' - ^^I'Ac Ao^r^rbducc 

2. to be able to recognise and state a WTlter*s attitude to his subject ; 

3. to be able to state a personal etpotibhal or moral reaction to the wnter*s treatment of 
his su^ect; ::i i:: i : 

.4. Cin^jsome questions) lo be able_to__compare and make a value judgment between two 
different treatments of the same subject; 

5. tb be able tb re^^hise ^lid cbmrheht ^h the accuracy with which literature reflects 
human characteristics, rdatPbris^d problems, e.g. the qiialiti^ and faults of characters ; 

6. to be able _tp recog?ilse and comment on the eiemeDts_of literature w hich ^ntribute to 
its eixioyment,,e,g, originality, humour* suspense,^ s^^^^^ dialogue, « 
etc. (This is expected in some other questions, besides the poetry ones.). 

Teaching Aims ' ; 

The implied teaching alms are: 

- ^ ^ ' reading shoiild fc« wide and based on personal inter^t ; ^ . 

2. that pupils should become experienced in recognising and stating their own reactions to 
attitudes and ideas presented by writers^ 

3. that they should be made aware of some of the techniques used by writers to create 
impressions. 

INTERPRETATION AND LANGUAGE 

To test t he candidates* understandings of the contentofal passage of Jiterary merit .o_n^ simple , 
level and also h is understanding of deeper or 1^ ijnm^iate aspects of tbe writer*s meaning. 
Also, to test knowledge of language,^ comprehension of the whole or a substantial part 

of another passage oi straightforward prose. ^ 

^ ^ 1 14 
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Criteria 

The 1968-74 papcrejiave required _ - ' 

1-. to exjiibit a breadth of ^Qcabiiary, by recognising meanings of words and by repro- 
ducing ideas in their own words; 

2: to explain the hbh-Iiteral rtleaxiings, cbnnotatibhs and associations of figurative 
language; 

3. to explain the reasons why the writer has used different registers of language; 
4: to reco^ise patterns used in the structure of the passage, e:g. statement - illustratibh, 
statement - contmst, opinion ~ reason, etc:; 

5. to recognise and reproduce in summary form the logical sequence of ideas in the 
passage; A 

6. to recognise and reproduce the sipiificant points of an argument, as o^ipsed to digres- 
sive, illustrative or secondary ones; 

1. lb explain 4he reasons for particular usages of punctuation or grammatical forms, or 
for particular types of sentence stnicture; - _■ 

8. tP bejabie to_useJh^eoatext of the passage to find clues to the meaning ci implication 
of various types of linguistic usage; > 

9. to be raiiiiliar with such linguistic cbncepts as are necessary to disciwS^e world ngs of 
language in specific contexts (hbiiri, adjective, verb, subject, complement , , ,). 

Teaching Aims 
The examination implies - 

1- that pupils should be experienced m 'close reading'; i.e. that the^^.should be used to 

responding to every means the writer has used to communicate his meaning; 
2: that they should have enough ^t^hnical* concepts to be ab]^ tb indicate what these 
various means are: 



Second ±isl of Skiifs 

After teachers' reaeti^^^ Jjiese 'objectives* had been sought in an 'open' 
questionnaire, it was intended ta try to survey the views -on S4 'objectives' 
Pf ^^J^^gg^r samp^^^^ The results of this 

second questionnaire are, like those of the first, reported in Chapter I: 

The need to write it led to a more detailed list of sicills than the brie based 
solely on P-grade examiriati oris. At this stage the work c>f several otherTesearchfers 
and teachers of English came to the aid of the research team. Some of the 
influences ar^ P^by were less direct. 

The researchers are aware lhat, as far as the Second List is concerned, they are 
indebted at least tb the following: 



James Britton and his colleagues at the London institute of Education. 

The writers^and compilers of the material used in the Open University's 
Readirig Develbpmerit Cburse. 

The Scottish Central Committee on English, for ideas set out in various 
'Bulletins': 

The authors of 'Assessing Cbmprehensibri', a discussibri pamphlet of the 
Lbhdbh Association for the Teaching of English (Blackie & ^on, 1968). 

F. P. Robinson, for ideas contained in Effective Study (Harper & Row, New 
York, 1961). [ 

L-^E_- W- Sm[tli, for ideaiJri Zo^ 1972), 
Anthony Adamsv and John Pearce, for ideas in Every English Teucher (Oxford 
Uriiversity Press, 1974). 

The followingjs ajlightjy^a^^ sent to the 

teachers: TTie actual questionnaire (with results) is printed in Appendix 1(b). 
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NOTE ^ ^ \^ 

SpcajajQg, Teading, writings aniih^^^ 

r LUictions_i nyolvi ng sets of jskiUs.jpersPjial characteristics and experiencc. M ost pieces of \\Ti ti ng, 
for instance,, wiii show _ that pupijs have exliibited severai- Pf the skills set out__in_ these, lists. 
The classification. .used will,, it ir hoped, cjarif^y what^performance' in. English involves^ but it 
would not be desirabJijLpr possible,^ to assess the achievement ofeach skill .separately. ^Further, 
the list js not eomprehensiVe. Some factors affecting English *perfoiTn such as t lie stage 
of intellectual and emotional development aj)upil has reached, cannot be accurately measured 
with existing testing tools. There are also, even at O-grade standard, qualities recognised as 
intrinsic to good communication - e.g^ breadth and aptness of vocabulary and variety and 
appropriateness of sentence structure - for which specific criteria- cannot be fonniilatcd in 
advari.ce. 

WRITING 

Tliree.main.h_eadin^ arc proposed, taken ^fr^ James. Brittp.n*s*W.hat*s the Use* in Lwtguage 
and Education (Routledge and Kegan Paul, for the Open University). 

1; ^Transactional' Writing: tP direct, question, get things ddne, p^^ . . 

2. *Expresslye*..w.riting: tp exchange pr reveal feelings and ppinipns, cpnvey attitudes, 

reveal perspnality ... . * " 

3: *Poelic' writing: to cpnstrucl linguistic artefacts prganised language - as a itieans 
of trying to explpre and master the cpmplexlties pf reality. 



TRANSACTfONAt' WRITING 

Pupils might Ih> expected tb; _ _i : : i 

t: Write instructipns fpr a simple task (recipe, care pf spprts equipment . . :): 

2. Fbrrnuiate rules fpr a club« social everiirig, dr spme situatipn in which grpup behavipur 
miKt be contrplled. _ _ 

3. Write nptes pn_ events as they happen. 
A. Report eyerils factually^ _ l ^ 

5. Write a repprt presejiting infprmatipn Ipgically. 

6. Order ed jmragraph ideas. 

7. Write an argument based pn facts .fpr pr against a ppint pf view. 

8. Present evidence and draw cphciusiphs from it. 

9. _ Give arguments for a/i^ against a ppint of view and draw rational conclusions. 

10. Illustrate a general idea with a particular example. 

11. Generalise from particular instances. . ^ 

12; .Develpp an idea, illiistrating, quoting, adducing evidence In support: 

13. Write a logical, theoretical argument, given premisses. 

14. Write advertising copy (e:g: for school events : : 

j5. Write a (Hsrsuasiye ar^uneht using emotive language to win support. 
16: Write a speech for or against a motion. 

17. Summarise fact u^ . 

18: Summarise an_ argument, maintaining the logical thread. * 

19. Write personal letters of varibUs kinds. 

20. Write formal letters to achieve, practical ends. :^ 

21. Use the appropriate register of language for audience and purpose. 

•EXPRESSIVE' WRITING 
Pupils shPuld be able tp : ^ 

22. Describe their pwn feelings abput br stimulated by events they haye experjenced. 

23: Define theirjittltudes to aspects of society which impinge uppn their experience (e.g. 

authority, other groups, other races): . *" 

24. State opinions on ^• : . : * 

25:' Show awareness of the causes of their feeling and attitudes: 

26. Convey |heir interest in various topics by witing; knbwledgeabl^ them. 

27. Write4antasy stories (in which their own feelings, wishes or desires are significant). 

28. Write iriiagihatively (i^. with brigthality) or critically, expressing a personal view, given 
ah artistic or haturar stimulus (film, pbem, music, picture, pViysical experience . . .). 
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TOET^C^ WRITING 
Paplls should Be able to: 

29. Write stones wtlvk)mccx^s^^^ th^caiises arid effects of events; 

3C: Organise events in a story to create a climax. 

31. Cn^te suspeh^ '^l^i^ofy- 

32. Write stories ia which the feeling of more than one character are revealed. 

33. Write stori^ wHigehuihc inteiplay between events and character: 

34. Reveai' character in a story through dialogue. 

35. Dramatise (riot just describe) events in a story: 

36. Write a story with a nioral or message. : _ _ : i: 

37. Write adopting the point of view of someone with feeling and attitudes different from 
their own. 

38. Write in simple poetic fphris (c.^. ballad, haiku, frl^ y^ . .). 
39: Write in stricter poetic forms (e:g. blank verse, sonnet . . .). 
40. Exploit (in prose or verse) the emotive overtones of words. 

41: Use language figuratively: 

42. Exploit the sound and rhythms of language to rtiake a stronger impfessibh on the reader. 

- - - - , ■ ■ 

Knowledge of appropriate vo^jilary and of a Vi3iriety of sentence stnjc is inherent in^ the 
objectives listed above jmd the skiU with which this knowledge is employed would be assessed 
in jnd^g_the^tandard oCany piece of writing t?y comparing it vith other w^ 'Technical 
correctness' would also be taken into account, Whil^^ it is not practical to specify in detail 
minimum crileria in grammar and spelling, the following list indicates some skills contributing 
to 'correctness*. 

Pupils in their normal writing might be expected to; 
43>^ Use fiill stops correctly in nearly eveiy instance: 
4^ Use the question mark cbrra:tj^ when j^uirc^ 

45. Use quotation marks fbr Direct Speech: . : 

46. Use the (X)mma cbrro:tly to mark off subordinate clauses and parenthetical or apposi- 
tipnai phrases. : 

47: Use the comma coirectly in combination with quotation marks. 
48. Use the cxclamatjon i^^^ correctly. 
49: Use the colon correctly. ^ 

50. Use the »mi-colqn corro^tly. 

51. Use the apostrophe correctly. 

52. Use quotat[qh marks CO ri-c^ for slang, foreign phrases : : 

53. Use parentheses correctly. 

54. Use the dash correctly. 

55. ^pejl correctjy. : :: i ::: : : 

' 56. Show evidence of command of various grammaticaLdevices^ suchas tcns^^^uboxdinate 
claus^, ji&rticiplw, noun phrases, hnking words, contributing to the complexity and 
variety of sentence structures. 



GENERAL READING 

As regards their reading, in q^rouVof school, piipils might be expected to: 

57. Reproduce the ^st of a narrative; 

58. State toir own eiTibtibhal ri^tibh to the ^bhteht of what they have read: 
59: Recognise or state the writer's attitude to his sut^ect, and some of his ideas. 
60. Recognise oj^state the Jbhe/fwling of what they have read: 

6 j . State their emotionaj and/or moral reaction to the >yriter's treatment of his subject. 

62. Reobghise and/or aSmmcht bri the accuracy with which literature reflects human 
chaiacteristics, relatibiishil^ arid problems: ^; 

63. Roco^isc and/or refer to the writer's techniques Ce g. devices for creating suspense, 
revealing character^ or stnichirally relating parts to the whole, his use bf figurative 
language, or other means he uses tii) make ^n emotional ahd/br moral impression bri 
the reader or to create an ijtiaginative world^^ 

64. Give reasons for enjoying what they have read in terms of the writer's skill. 

117 \ 



^A^^ '^o^ that one piece of writing Is superior to another. 

66. Locate specific infonnatioain a bopk using Contents, Index, Chapter headings etc. 
67: *Skim' read to get a general impiesiidn of a passage. 

eOMPI^HENSIOI^J 

When carrying out 'close reading* of a piece of writing, pUpils might be expected to: 
?^Sy^J°I ''^lU?L^P™at^ ^pjicitly stated. 
69. Paraphrase or translate such inforaation: 

?^*^T^^^^_?as«dj)n info explicitly stated and on their ovm experience. 

71. Recpfi^isc pr state the niain pointis presented, as opposed to digressive, illustrative or 
secondary ones. 

^P^'!^!^-^^4-P^^P^^^'^^^ the main ideas praiented. 

73. M^f^C'^jSSam^ni based on their own Icnowledge. experience and v^Ues about the 

"^udity of ideas in a test. ._ 
74: Pistingolsh factual writing frbiri bpihibn or persuasion. 

75. R«?pghise or state jhe effect on meaning or tone in context the use of difierent 
re^stcrs of language. 

76. Recognise or state thc^ effects in context of partioUar sehtehce structures. 

77- ^^_g?is?_oritate the effects in context of pardcular usages of^rammar and punctuation: 

78. Explain the impUcatlons. oyerton^, connotations in context of Jangua^ used figuratively . 

79. Recognise or state the general idea reprinted by a particular inddent, detail or im^: 

j^pils'Jknowlcdge of language Jirig^it be expected to include the foUbwing concepts; i.e they 
might bc_cxpcctcdJ:D be iible toirecognise or explain the function of each in context, even if 
they are not familiar with the terminology: 

80: Noun. . 

81. Verb. 

82: Adjective: 

83. Adverb. , 

84. Pronoun: 

85. Sentence. . " 
86: C3ause: 

?7- Adverbial clause. ^ 

88. Adjectival clause: : 

89. Noua clause. 

90. Phrase. - 

91. Subj^t. . . 

92. * bbjecl/CkjmpIement. 

93. SinguIar/PIUral. 

94^ Tenses. _ _ : : 

95. Fast and Preasnt Participles. 

96. Finite/Infinite moods. 
97: Subjunctive tnbod. 

98. Active and Passive voices. 

* ^raftTimfor u CriTerton Test 
For assessment in the 'comprehensive testing procedure*, pupils might have been 
asked to complfte a^^^ which together would have tested ali 

the items in the Second List of Skills. : 

Possible Criterion Test Tasks 

WRITING 

Pupils might be asked to : 

(1) Write factual reports, orgaJiising material logically (Skills 1-6). 

(2) Make a case for a view, present an argument (7-13). 



. ' \ 

(3) Write persuasively, using emotive language, rhetorical devices ... (14-16; 
40-42). . \ V 

(4) Summarise facts and arguments (17-18). | 

(5) Write letters for various purposes (19-20).. ■ 

(6) Write to' recall events and atmosphere of past experiences (21). 

(7) Write to communicate personal feelings, opinions, attitudes, fantasies 
(21-27). 

(8) Write stories (26; 28^37). 

(9) Dramatise (33-34; 36). 

(lb) Write poetiy (39-42). ' • 

In al[ the above the following would be taken into consideration in an 
assessment: 

Breadth and appropriateness of vbcabulai^. 

Appropriate use of fbrmal/inrbrtnal, emotive or figurative language, and of 

the sound and rhythms of language (37; 40-42). 
Organisation and paragraphing of material (6). 
Correctness of spelling (55). 
Correctness of grammar (56). 
Correctness of punctuation (43-54). 

Complexity, variety and appropriateness of sentence structures (56). 
READING 

As far as reading complete works is concerned, pupils might be asked to: 

(1) Reproduce the gist of what they have read (57). 

(2) State personal reactions to the content and to the writer's, treatment of 
it (58-61 ; 65). 

(3) Comment bri characters, relationships, problems portrayed (62; 65): 

(4) Comment on style/techniques (63-65). 
They might also be asked to: 

(5) Keep a record of all their reading: in and_ put of school le^g^ mpiithly). 
Checks could be made to counteract cheating bx regular random ques- 
tioning of pupils bri the books they have entered in the record. 

As far as /ciose reading' is concerned, pupils might be asked to: 

(6) Recognise, recall,^ translate, explicitly stated information ('Literal com- 
preherisipri') (68-69). 

; (7) Make inferences based bri the text. ('Irifereritial Cdmprehensidn^) (70 ; 74). 

(8) Make judgments about the validity of ideas in the text. ('Evaluative 
Comprehension') (73). • 

(9) State the effects bri riieanirig arid tone of various aspects of the writer's 
skill (74-79; 80-98) CAppreciative'Comprehension'). 

(lb) Suniriiarise the main ideas in the text (71-72). 

The firstAbluepririt' for the Cri was designed to include all these 

tasks, thou^ some were^rouped together in feasible test papers, e.g., as 'Factuar 
or Tersuasive' writing. The Criterion Test at this stage was shaped like this : 
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First Craft Ftanfor Criterion Test 



WRITING -__ - _ N 

, (1) Expressive C 4 from 6 or 7 opUons. 

(Possible—. Picture and vero^ stiitMi. 

Cnteridh Tfest Completely 'bpeh* rubric - *relevaii<** to be jnterprcted very broadly, 
tasks Nc^; (6) Credit to be ^iven for revelation of personality, individuali^, cohfi- 
and (7).) deoce, sincerity . . . (judged from the way the language <s used). 

Likely cdntent: personal experience, exploration of feelings, attitudes, 

their ^uses and eB^ts. 

Mi^t be in narrative, d^criptiVe or reflective forin: 

(2) fhctuai -i 1- from 4 options; :: : 

(Tasks Nos. (1), Probably verbal stimuU only, thou^ piclore possible:: : 

(4), and (5).) &edit:lQ :bc:givcn fan relevance, orgarrisatiott jof Jisterial, cl^ty, 

appropriate lay-oat and style lor prospective jiadience and funcdon. 

Snort: piece or writing. Mi^t be in letter or report or ma^sne 

article form. • 

(3) Pmuasiye - 1 from4 cjptioiis. 

(Tasks Nos. (2), Picture and verbal stimuli^ 

(3), (4).) CrwJit ti) be ®ven as^i^ (2), and for abiUtx to use evidence, impress 

or mpyejby emoUye i«« of languaw, jllu^^ 

Short piece, possibly in form of school magazine article. 

(4) Poetic Story/Fiction forjiterary effect. _ _ 

(Tasks Nos. (8), Unlimited time (e.^. oyer a two or three week^riod). 
(9), (10).) Credit for ; charactensaUdn^stmcture, plot, suspense, etc. . . 

Might be incomplete when assessed. 

Stimulus to be the banning of a story, written by the Research 
T^eam (^ options), though pupil wouJd be free to make his own start 
if he wished. v 

READING 

(1) Rtsponseto a. *denerat* qu^tions on Reading, similar to tfac»e set in O-grade 
Literaiure Paper_t:i:_ i:z :::_::: :_:^ ii: _z: :: :l 
(TJ9isksXl) - (5); b. 3:texts_provided±>ySCRE::onc short story, one poem, one self- 

involving contained extradJrom^play^ :_:: _: _ i l: _^_:_ ::_:ii 

also tasks QuistLQns_on eacl^d^igned tat^^t 4^mple level .appreciation 

(6) - (10).) of Jhese texts as^literatur^-l.e^jiucstiojQSLinidj!^^ between very 

spccjfic onss rcfejxina^o pajlicular_ words/phr?iscs,^^^a^^ 
g^eral ^nes, which seem to encourage re^urgitatipn pf content. 
Credit^ e.s.identifijpatipni^^ 

ter*s behaWpurt pei^ (the criteria for 

credit depend on the specific material used). 

(2) Comprehension 3 or 4 passages with qu^tlons covering all coxriprehehsion skills, 
(Tasks (6) • (10^.) including; sunimary. 

(Mj^t also include a test of awareness of the way language is used 
in advertising.) 

There was, in addi tion the possibili^ of subjecting the pupils to a standardised 
yoeabula^testan verbal^reasoiim^test.F tests o f sp elling, punctuation 
and grammar could also havcbeen devised to complement the assessment of 
the pupils' writing. 

It was th[s scheme for a dauntingl^ massive test which was matched against 
reality in the form of the testing material which had survived pre- tests and expert 
evaluation, -the expected limit of pupils'^ endurance, lhe/"time schools would 
aHdWi^ the teMce available^ ajid the need for C3-grade examination pr^tice. 
To make it fit better, the following decisions were taken : 

1 . *Poe!ic* writing would be dropped. The schools were unlikely to be willing 
to accept the imposition of a three- or four-week long _ex_ercise_(except 
for the one school in the experiment where such an assignment was- 



fidfmalj. There wdlild be cyfficulties in finding time arid mpriey for Re- 
liable marking. Pre-tests in other 5chouls had produced a little very 
interesting work, hut seemed to show that success in writing fiction 
requires a kind of teaching which was not prevalent, at least in those 
particular schools. 

2. There would be no standardised tests or foVmai tests of *correctness'. 
With limited time, it^ a low priority M te^^^^ 

sbihethirig other than normal performance in school or examination 
contexts. 

3- The flumber of o^^^^^ 

would be reduced to two or three each, and verbal stimuli only would 
be used. These decisions were taken for, two reasons: (IJ m order to 
reduce the range of writing possible, so that pupils could not easily slip 
into 'expressive' writing through misinterpretatidn of the stimulus, ^d 
so that markiiig cbu^^ more easily standardised; (2) because test 
stinmli successfully pre-tested and/or approved by the consultants were 
in short supply. < 

4. The number of options in 'Expressive' writing was, by contrast, to be 
increased, to allow as much freedom as pj5ssible to pUjjils in that section 
of the Test. Picture stimuli were tq*be used for the same reason. 

5. One story only would be included iji the 'Close Reading' test described 
under 'Response to Literatire - bT The rembyal of poetry and drama 
from thjs test was to save time: A short story seemed the most appro- 
priate text for a wide ability range. ^ 

6. For the cdmpfeh adv^tage wouW of the^Research 
Officer's involvement in the SGEEB 5ub-coinmittee on multiple-choice 
testing for 0-|3:ade. Fassages and multiple-choice questions validated in 
SCEEB pre-tests of material developed by the sub-committee would 
complement a 'traditional' interpretation paper, which would, however, 
be desi^ed sb_ that the secpn passage sought specifically to test aware- 
ness of structure in the writing. 

7. The first ^ay's papers of the two-day Criterion Test would follow the 
pattern of the O-grade examinatibh, so that it prbvided some examinatibn 
practice for pupils whose schools had not set 'prelim' examinations: 



U. The Criterion Test 

The Griteriori T^st fma^^^^ five j)ap_erj^ lastjn^in alL some 7J hpurs:, 

Ss elements and the skiils they were each intende^d to test are shown in the 
fq 1 Id wi rig table . The a ssigftmeri ts ari d q ues ti dri s set an d the riiarkirig^ criteria 
applied can be scrutinised in detail in Appendix 4, where the test papers and 
marking schemes are printed. 

The papers >yere taken by the pupils in numerical order, but are grbuped 
here under 'Writing' and 'Reading' so that the^kijls assessed can be considered 
in cohesive- grbups It: wi U be: seen that ihe Griteribn TesLbbliged the pupils 
to show the extent of their command of a much wider range of skills than any 
one pupil needs to eall upb^n iri^^^^^^^^^^ The Criteria 

T6st was, however, not absoldtely comprehensive and it still allowed sorae-^ 
choices of task, sd that pupils dbtmhirig the ^ papej^may^ch 
have been assessed on (at least some) different skills: this phenomenon is, of 
. cotSrse, even more likely to occur iri both the O-grade and any internal assess- 
ment scheme, and contributfes to the imprecision of discriminatory assessment 
in English. 
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The Crirerim Tesi : Descriplion oj Papers Set and Skills Sampled 



WRItlNG 



PAPER 1(A) _ e6nipGsiiioiu 1 hour _i 
StimulLF^pictur^quoJatioxis^till^ open 
to variaiis^iiiterprctatiQns* intended to __ _ 
allow as rojLich_fR«dom_as possible to the 
pupil to write jLbout something which he 
knows and cares about, ' 

InsLriu;tion__to. write _a__co^ 

revealing thoughts, feelings, experience or 

ppi nipns._ 

IS minutes allowed beibrc start to make 
choice of topic. 



Sfcffts (nambers refer to Second ttst oJ SkttIs] 
' 11 : see page 114) 

(SojncofX - 1 11 — : . 

Describe. Qwn^experienccs, feelings (22). 
Definej^wjiJiUiUides (23). 

Express opiDLons C24)._i ^_ : 

Explore causes of feeling and a_ttitudes_(25i._ 
Exhibit interest in and knowledge about topic 

chosen (20. 

Reyeal i ndiyidual i ty , i magi nati veness (28), 
perhaps in fantasy_stpries i^ involving 
own feelings, wi^es, desires. 
(Possibly) story-writing skills (29-37)^ 
Cornihand of vocabulary (not itemised) and 

sentence structiirw (56)j ^ 

Selection and organisation of material (6). 
Techmeal cdrreethess (43-55). 
(Possibly) rr.plbitatiGri 6F figurative arid 
crribtive lah/uage (40, 41). 



PAPER II 1(A) Factual Writing. 

30 minutes 
One from (wo options: a specifically 
defined writing (ask, with indication in 
the question of the 'purpose* and likely 
*aadience': 



(Some of) 

Write instructions for a task 1^). 

State rules of sport or gaJtle (2),: i . 

Report, pr^niuig infbrmatioh logically (S): 

Stimmarise factual infbrmation (17): 

(Perhaps) illastrde general idea with 

particuIaEiexaraac (10); - : 

Qrderiand param^ph Jdeas (6). 

tJse^ppropriate register (2J). 

Commandj3f vocabulary (not itemised) and 

sentence^ structiires_(56) 

Technical correctness (43-.^5). 



PARER ni(B) 



Persuasive Writing. ' | . 
^ 30 minutes ' 

One from three options. Specific task; 
with :'aadierice' and 'purpose' indicated, 
the latter being to infioence others* - 
actions by effective arguments. 



skms 

ifSome of) X. 

Argument bas^ on tots (7^. 

Draw conclusions from evidence (&):i 

Give arguments for and against and draw 

conclusions :(9): ii:i : ii iiii _ i__ __ 

Useiemotiveiiangnage,^ rhetorical devices to 

strengthen^rgument 

(Perimp^ write _eiFective_advertising copy (14). 
Illuslmte idfi^iwith examples CIO). _ 
(Perhaps) generalise from particular 

instanpes CI !)• " 

(Perhaps) deye jop an idea, illus t rat ing, 

qupLing«_adduci_ng_evid(^^^ 

Order and paragraph ideas C6). 

Use appropriate rejgister (213. 

Command of yoqibida^ (not itemised) arid 

seriterice structures (56). 

Techriical correctriess i33-55). 



PAPER 11 Interpretation imd I:anguage. 
1 hour, 35 rrtinats 
Passage 1 (ftom *To Kill b Jyfockingbird* 
by Harper Lee) 



2^ 



(AUoQ: / 
tlter^comprehension (68): 
"Translation: C69);i 1 1 11 1 :: 

inferential comprehension (70). 

Awareness i>f Jmplu^tions of register or 
tone of language used (75). 
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Passage 2 (The Excavation of Ur* 
by C. W. Cdraxn). 



(AJI^O- :- 

Literal CQinpcehension (68). 

Translation (69). 

Inferentjal comprehension (70). 

Summary (71-72). 

Appreciation of effects of sentence 

structures ^7^. 

Appreciation of effects of usages of grammar 
and punctuation (771. _ 
There was also conccrh to test awareness of 
i means _us^ by the writer_to relate 
paragraphs to one another structurally (63). 



PAPER IV Comprehension (Multiple- 
- : choice). 1 hour, 15 minute 
Passage 1 (*F£dce Antigues') 



Passage 2 (*Natasha*) 



Passage 3 ('Television') 



(AUbf)- 

Literal CdmpFehensibii <68). 
Inferential Cbmprehcaisiba (70). ■ 
Recognise main^oints (71). 
Recopiise impli^tiorus f<}r-me^ of 
. register of language iised (75): 

As_Pas$ag|s j » plus _ 

Recognise tone (60) : ^ ^ _ 

R«»jstdse 1^ for revealing 

character jMd (H^tinK^^^ _ 

Recognise writer's attitude to sul^ect (59). 

o 

As Passage 1 . 



T^eUmem - — 

'PAPER V ' *aosc Ridding' of Short Story 
(unlimited time) 
Hie story was read aloud to the pupils, 
wfao^eacfa hada copy of jt. The 
instructions asked them (1) to re-xead it 
with certain43urposes: in mind; (2) to: : 
answer ^nJheiiLh^ds'-^omeL questions. : 
desi£p3^edto_belRlheir_understandingi2f_ it ; 
(3) they were^ tlien aUo^^ fwithin reason) 
unlimited time, to write answer to Jthe 
assessment questions, which had been 
deriycd_directly_ from, the purposes 
suggested in (1) above. 



Skms 
(All bO 

State bWn embtibhal and liibral ri^tibh to 
the jtoiy (61): _ 
Appreciate and comment on the feelings/ 
problems of the characters (62): 
RecQ^ise the^ writer's attitude to the 
chamctei^ and events in the story and 
sho^ that it influenced own reactions to 

thein^9.X - - 

R^l^lsei?r state techniq ues josed isy :the 
writeiLtD revealidiaracter, Jhint^t^sociaL L 
backgrauttd, prepare reader for the climax 
of the story . . . (63). 



PAPER I(B j Genera j Readi ng Questions. 

_ : L hour, 15 rninutes 

I questions, one from each of 3 sections^ 
Prose, Poetry, Drama, each containing 3 
options. 

*Opcn* questions, no texts sj^ified - 
similar to 0-grade examination questions. 



(Some of) :i 

Reproduce^sLof a narrativei57j\ 

State personal reacjions to content (58) and 

to_writer's treatment of it (61), 

Recognise emotions in fictional characters 
and show understanding of relationships 
between them r62)^ 

Recognise and refer to some aspects of the 
writer's style/techniques (63). 
Appreciate tone, mood, atmosphere (60) 
Cm Section 111, Drarna, question 1). 
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Skills mt dovered ty ihe Chmion fesr 



Items iricluded in the Second l^fst af Skills biit excluded from the Criteribri Test 
were: 

WRITING: Skills not tested 

3. Write rfotes dri events as they happen. 
1 3. Theoretical argument. 
16. Speeches. 

18: Summary of someone else's logical argument. 
[9 Sc 20. Letters (most relevant skills were tested in other tasks). 
38 & 39. Write poetry. 

Though somi^pUpils might have iattempted isome of the fbllowing 
skills in the jCriterion Test (and some did), it was not intended 
that the Criterion Test require pupils to show them: 

Skiiis usable in the Test but not required 

9. Argue for and against a view .and draw rational con- 
clusions, 

10. Generalise from particular instances. 

1 1 . Develop an idea in a sophisticated ^ay. 
29 - 37. ' Write successful literary stories. 

40 - 42: Exploit emotive, figurative, rhetorical qualities of 
language. 



READING: 



Skills not tested 

65. Give reasons for theiudgmerit that brie piece bf writing 
is superior to another. - 

66. 'Survey' skills - finding information in a book. 

67. -Skiril' reading. 

73. MaRe judgments afc^^^ validity bf ideas in a text 
(evaluative comprehension): 

74. Distinguish factual writing from opinion or pereuasion. 
80 - 98. Show specific knowledge of various grammatical con- 
cepts. 



Skiiis usabie in the resi but not required 

59 & 63. Comments on writer's attitude, style-andiechnique were 
required^ but at a fairly low level bf sbphisticatibri. 

64. Give reasons for enjoying what has been read in terms 
of the writer's skill. 

78. Cbmriierit bri figiirative language. 

79. State general, idea represented by particular image, iri- 
cident or detail. v 
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III. The Marking of the Criterion Test 



U) DouMe-piarkin^^ ^ _ _ ^ . . _ 

The researchers begaruwith the intention of adopiing multiple-markings brii the 
recpmmendatloh of_*SchboIs Council Bulletin No. 12, Multiple-marking of 
English Compositions'^ by James Britton, N. Martin and_ H, Rosen. The_c^^ 
of triple^markirig having been found greater than the project could afford; 
. doubie-marking was decided on, its clear superibrUy over single narking being • 
cbrifirmed by a study conducted by Robcrt3Vood andrB. Quinn o he University. 
Qf London School Examinations Bbard\_The_re wM noJ^arker-Unreliability in 
Paper IV (multiple-choice compreiiension tost). For Paper II, Interpretation 
and Language; the mbstexperi^^^^ were chosen to carry but single- 

markirig, the marking team of eight containing three of the Principal Examiner's 
team and the two research bfficers. 

(2) Fersomel 

A team of teachers was selected by the researchers from lists provided by 
^CEEB of those with good or. at least, satisfactory records as O-grade markersi. 
Three criteria of sejectioniwere emplb^ed : (1) tbat the marker shouldhave marked 
the relevant O-grade Paper satisfactorily for at least twoyearsj £2) that he should 
not teach jn brieqf the project schoolis ; (3) that he live within fairly easy travelling 
distance i)f Glasgow or Edinburgh. Most of thpse emplc^ed were regard^^ 
the SCEEB examiners as fujly consistent markers; a few were only adequately 
consistent; and a number tended, ta severi^ prjenienc^, but could ha^^^ 
marks adjusted by marker-standardisation. The majority had several years' 
experience as markers. Thrieecuriient memb^^ Principal Examiner^ 

dardising team were in the group, jvhich could be said to contain most of the 
• best O-grade markers resident in Strathclyde, the Lbthians. Central Scotland 
and Fif^, with one outlier in Aberdeen. In all, 36 teachers were involved, in- 
cluding two of the researchers whb were experienced examination markers: 

(3) division of ±abour / : : _ i : 

Papers in the Criterion Test were assigned to markers as follows. 

Fa^er I {A and Composttion und Feuding) 
First marking: : 8 markers 
Second marking: 9 markers 

Faper J J {interpretation imd Language) 

One marking only: 8 markers (including 3 examiners and the 2 
researchers) 

Faper III {Factuai and Persuasive Writing) and V (Short Story) together 
First marking: 8 markers 
Second marking: 8 markers 

Paper IV (Comprehension, fnuttipte-choTce) - marked by computer 

(4) Procedure _ _ : 
Since the Criterion Test was intended to perform the Jame f^^ 

O^rade exaiTiinatibn and test some other skills a^s well^thc marking procedure 
was kept as close as possible to that emplc^ed by the SCEEB. Marking instrUc- 
tic>ns were written for each paper by the setters (i.e:, the researchers) and these 
were distributed to markerSi^ wUh phpTocbpie^^ 

markers' meetingis. These meetings were, however, decision-taking, unlike the 
' Wood and H 976). ^ • 
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SCEEB maricers;_mcc group of SCRE markers set its owii stan^ 

dards for the sample scri,>ts and, in the case of Paper II (Interpretation and 
Language), the marjcirig in^tructioris were modified as a result of the group's 
discussions. The Second Marking took place some three months after the First 
(the O-gradc having intervened). One or two markers and the researchers^ who 
conducted the meetings, were involved in both First and Second Markings. 
Almost all marked the 1977 O-grade examination. The irifluence_of these factors 
rnaj^ have been the re^^^^^ why there was very little difference in. the standards 
set by First and Second Markers on the isamplc scripts. For the Paper 1 sample 
scripts, the First Marking m^^ was 30.7/60; the Second Marking mean 

score was S\. \/£0. For the Paper IIJ sample scripts, the First Marking mean 
^corc was 1 1.6/20; the Second ^M^ $cp_rc was 1 1.5/20. For the Paper 

V sample scripts, the First Marking mean score was 10.5/20; the Second Marking 
mean score was 10.6/20. 

This apparent unanimity of standard on sample scripts was not, however, 
maintained bctv^^een First and Second Marking over all the candidates: the 
matter is discussed in Chapter VII. 
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Apiiendix 4 

Criterion Test Papers and Marking Schemes 

SGEEB/SGRE O-grade English Project 
'Criterion Test' 

PAPER I ' ' _ : 

A - CpmpositiDn - 1 hour 

INSTRUCTIONS In the following page^ you will find some quotations, i?ss^y 
titles and pictures. 

Write GNE composition showing what you have thought 
and felt about^ qum prtitje^r picture or, for instance, 
a quotation and a picture together. 

Your composition might be a story, a description of your 
own^xperiOTce _a^ statement of your yiewsj 

or anything which wUi show what you have thought. (Do 
: not siniply describe what is in a picture or what a quotation 

says.) 

You have 15 minutes to make your choice before the 
test begins. 



No. 1 



No. 2 



No. 3 



No. 4 



No. 5 
No: 6 



'I am alone with the beating of my heart.' 



'Behold, this dreamer cbmeth.' 



'I remember, I remember . . . ' 



'Once I am sure there's nothing going on 
I step inside, letting the door thud shut. 
Another church: matting, seats, ^nd stone, 

And Httle booksLspra pf flpwerSi^ cut 

For Sunday, brownish now; some brass and stoft 
yp M tM holy^end ; the small heat p^ 
And a tense, mtisty, unigoorable silence, 
Brewed God knows how long. Hatless, I take off 
My cycle-clips in awkward reverence.' 



Threat: 



Getting away from it all. 
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SCEEB/SCRE 0-grade English Project 
'Griteripii lest' 

PAPER I 

B - Reading - 1 hour 15 minute 

INSTRUCTIONS Answer THREE questions, one from each section. 

Note: In each answer you sh^^^ the title, arid author of the book, 

poem or play you are writing about. 



Section i: Prose 



Answer ONE of the fdlldwirig: 

(1) Shpw hovv a story or a novel you have enjoyed succeeded in erippihe 
your attention. 

T^^ur Misw^r sho^^ the story and also show why 

, you found' it interesting or exciting. / 

(2) Name a character in a story or hovel whom you have felt you feally 
uhderstodd.. 

Briefly describe the part pl^ed in the story by the person you have 
chosen. Tjben explain why the character appealed to you. / 

(3) Descnbe briefl^he contents of a non-fiction book yoa^ have found 
interesting or usefiSl. Then explain what features of the/6ook specially 
contributed to i^syihterest or usefulness. / 

> You might discuss the way the material was presented as weil as the 
content itself. / 
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Section II Poetry 

Answer ONE of the folldwihg: 

(1) Choose a ^beni which has excited pr amused br^ 

briefly what it is, about and explain as clearly as you can how it made 
its effect on ybii. 

Tptr milJit refer to tjje ioeas in^ t^ 

presented, mention any unusual or specially appropriate words, memo- 
rable images, etc. 

(2) Choose two poeras you have read on similar subject_s/_Sayiwhat_ e^^ 
about and what you like best about each. Point out any difTerences in 
the ways in which the poems present their subjects.; 

_ Ypu_may _cho^^^ for two poems, but the 

following list may give you some ideas: 
birds or animals; 
machines; 

human qualities, such as skill- bravery, strength,_endurahce, etc. ; 
feelings, such as fear, anger, hatred, love, envy, etc. 

(3) Choose a^pem whjch Mls a story - it may be written as a ballad or in 
another form. Give a brief outline of the story and then explain hpw 
the reader's interest is held. 

You might consider how suspense is created, description of character, 
imagery, incidents, etc. 
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Section ill: Drama 



Answer ONE of the foliowing:^ 



(I) Briefly describe a seme in*a play which you have read and would like 
to secipn stage.iTheh say how you think the scene should be presented 
so that the audience will enjoy it to the full; you should write your 
remarks under at least two of the Following lieadings: 

Movement: how slwuld M on stage? in wAat 

way should ihey come on or go off* at any particular points 
in the scene? 

Cosrume: what cidthes sHo^^ individual characters wear to emphasise 
^ their personalities? 

Ugli/ing: how should the scene be lit to help give the right mood? 

Speicfj: how should individual characters speak? Give your reasons. 

Setting: briejly describe a set which would help to create the right 
mood for the scene, 

(2) Choose two characters who come into conflict in a play or television 
serial you know. 

Explain briefly how the conflicLcbm^^ Then show why it is 

almost bound to happen, considering the personalities of the people 
involved. ^ 



O) From a play idu have read or seen 'performed, select a cfaaiacter who 
either suffers misfortune or experiences happiness. Say how far you think 
the misfdrturie or happiness was deserved. 

YdU shduld describejtw-iefl^w^ happens to the character in the play 
and then sho^ what kind of person he or she is: 



A 
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r c SCRE/SCEEB 'CRITERION TEST' 

PAPER I V 
Guidance for Markers 

General Foints 

: 1. Please do not write anything on the scripts-, or make arty correciJoii ma^^^^^ 
on them: simply record your scores for each pupii on the form provided, 

2. All composition -Sd reading answers w'il be marked ^ice: it is enough 
to jgiys an impression mark without <w d 

quah'ties or faults in the answers, though the guidance given below should 
be kept iil itiind. 



A. COMPOSITION - 30 MARKS "\ 

I. APPROACH TO MARKING: The marker^s approach toji con^ppsition 
should be_ pg si tiy e ra ther than negative . His jirs t task js' to assess th e 
positive merits of the piece of writing. 'Only after due cprtsideration has 

the extent to which the errors revealed during^ reading damage the writer^s 
power to comthuhTcate clearly and forcibly. The final mark will therefore 
be an assessment of the positive merits of the composition mo/Jified by 
an assessment of the damage done by erroris of grammar, spelling and 
punctuation. \ 

In an attempt to give s^e guidance, a suggested grouping of marks 
into categories is given iiiyParagraph IV. This divides the range of rtiarks 
into six broad categorie^^d suggests the features which should charac- 
terise the compositions falling within each of the categories. The categories 
are broad ones: the precise mark given to ah^ essa^ within a ca^ 
wij^dgpend on the marker's assessment of the degree to which it meets 
. _ the requirements of the category. 

^""^ It Jiiust be stressed^ however, that these categories are drily generalisa- 
tiqns for guidance which take no j:ognisance of errors in punctuation,^ 
spelfirifi and grariimar. When, for example, t^^^ relevance of 

the ideas and handling of language^ would put a particular composition 
[ri one category, bu th^^lpresenee of faults p£grariima^^^ 
spelling makes the marker feel an adjustment putting .it in a lower 
categffl^^ i s necessary, he w i 11 pbyi o u s ly make such an ^dj U s tmen t . Siich 
adju^ments should normaUy not exceed three marks. 

/Where a composition is so very Wjeak in some important respect that 
L4 writer's ability to ebmm^ 

ha^FTO^ abandoned. Such a composition wiiinot merit a pass whatever 
its good futures ihay be. For example, a consistent failure to write in 
sentence foSo should be treated as a particularly damaging defect which 
caniidt be recfemed by any of the other ftjatures. In such cases the final 
mark will be pnriiarily a reflection of this defect- Such exceptional treat- 
ment, however,^ should be reserved only for compositions where gross 
errors in expression seriously itnpede commwiTcatTon, 

No attempt should be made to allocate specific numbers of marks for 
specific aispects of th? composition^ Such a system often leads to a. grand 
total quite but of prbpbrtibri tb the value bf the wbrk as a whble. 




X^e range of marks s^^^^ it frequently is in some 

schools. Generdiy, the best pupils should be expected to make high and 
dccasidnally full marks. 

__ Marfers who nbrrhallyLina on a system where JO or 20 is the maxi- 
mum should bear in raind the need to make the mental adjustment to a 
system where the maximum is 30. 

itl. Length. No precise length of composition Js specificaiiy demanded in 
th^! paper. The actual number of pages wiU^of coursej depend: on the size 
of the candidate's handwriting, but between two and three sides can be 
taken as an acceptable length, assuming average size of liandwr^^ 
Cprnpositibns which fall signiiicantly below this should be penalised in 
proportion to the extent they fall short. In general such a penalty shouTeJ 
-^^t.^/'y/?^ /fe/^/^/*^ bejow the pass line if the composition is worthy of a 
pass otherwise: Compositions which are ludicrously short should, of 
course, receive a fail mark. 



III. NOTEON 'RELEVANGi^ were chosen to 

give the pupils a very wide choice of topic and type of writing, with a 
view to allowing them to write as freely and 'expressively' as JHey wished. 
The relevance of the writing to the stimulus chosen should not therefore 
be a significant factor in the assessment in this part of the test. Markers 
shQuii however^ consider 'internal relevance': their judgment of the 
quaiity of a composition should have taken into account the appropriate- 
ness of the content of the cpmpbsitibn to its own topic or theme; they 
might, for instance,^ penalise a composition which lacked a unifying 
theme arid cGnsis|ed^ ideas^ pbseryajipns or events written 

down without narrative or logical connection between them: 



IV SUGGESTED CATEGORIES FOR COMPOSITION WRITING. Two 
sets of categories are offered: the first is that provided by SCEEB for 
O-grade markers; the second is a less detailed categdfisation which defines 
the qualities of scripts. a little differently, though it does subsume the 
SCEEB instructions: Markers may use either or both. In any case^we 
shbuld^ be^lad_ to h^^^ the views of the markers on the rela.ive merits 
of each categorisation . 

SUGGESTED CATEGORIES FOR COMPOSmON WRiTiNG 
(SCEEB). In the fpllowirij; categories the characteristics of each have 
been listed under the two headings, 'Content' and 'Form and Style'. 

UlLcler ^Cbnjent' the to assess the quality of the Ideas 

and their relevance to the chosen topic: Under 'Form and Style' the 
rilarker is asked to asse^^^^ positive cpritributipn mad by such things 
as the choice of vocabulary, the choice of sentence structure, and the 
ordering arid pafagraphirig of ideas, to the effective expression of the 
content. . 

in practice^ it is almost impossible to separate these two headings 
when riiarkirig air essay, larid inarfcers slip^ neveri under any circum- 
stances, award separate allocations of marks to them: It is difficult to 
present any level of concept cdriviricirigly without a cbrrespbridirig level 
of stylistic ability. Thus in composition marking it is nonnaliy found 
thatthe level of a candidate's peffpfmance under brie heading is riiatched 
by the level of his skill under the other. 

Markers should note that the heading 'Form and Style' refers to the 
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•assessment of the positive merits of expression: Errors in grammar, 
punctuation and spelling should be dealt with separately as outlined in 
Paragraph I: M 



CATEGORY F (27-30) ^ FIRST CLASS COMPOSITFON WITH REAL 
SPARKLE 

Compositions in this category will be distinguished by a sparkle in the 
handling of the topic which reveals spmething of the personality and 
fbrcefulness of the writer. Note, if a well written composition reveals as 
much maturity and forcefalncss as can be reasonably expecte^ from an 
P-grade Candida tCv there is no reason why it should not be awarded 
^ full marks: 

Content Compositions will cover chosen topic very fully and show sin- 
cere in terest in an d kn owl edge of t he s u bjec t . The pb i ri t of t h e topic w \\\ 
be clearly grasped and all ideas will be relevant to it. In a story there 
will be a gripping and interesting development of character and situation. 

Form and Style Compositions will be clearly paragraphed^ with logical 
sequence of ideas. Th^ sentence s^^^^ ideas forcefuliy 

and the choice of ^vocabulary will be wide and apt: The composition wil! 
reveal a real arid forceful coriiriiand of lariguage. 



CATEGORY IL(23-26)- VERY COMPETENT COMPOSITFO WITH 
VERY FULL DEVELOPMENT OF SUBJECT 

Compositions ill this category will show a development of the chosen 
tppic_ \yh ich i s bo t h_f y 11 and re l_e y ant . However, _th e^ con ten t - an d sty 1 e wi ! 1 
lack those recurring touches of forcefulness and personality which mark 
but a Category I cbmpbsitibri. 

Cbiiteht Ideas will aU be relevant to the point of the topic arid of a high 
—standard. However,, they will not quite have the depth or sparkle of those - 
in Category I. They wiH sufficient in number to give a very full treat- 
ment of the topic. In the §tbry there will be a very competent development 
of character and situation. 

Form and Style the composition will becJearly paragraphed, with logical 
sequence of ideas. The command of su itable senterice^structures^arid c^ 
of apt vocabulary will reveal a very competent command of language. 

CATEGORY III (19-22) - COMPOSITFON WITH A REASONABLE 
DEVELOPMENT OF SUF^JECT 

Compositions in this category will show a reasonable number of relevant 
ideas. However, th^^^ the range and fiHlness, of C FF, 

and the sparkle and forcefulness of Category f compositions: 

Content There will be a sufficient number of relevant ideas to give a 
reasbrtable development of the topic. The ideas will be riibre pedestrian 
than in the first two categories. In a story the development of character 
and situation will be reasonable but lacking the liveliness, freshness and 
fullness of Categories I arid TI compositions. 
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Ppf^iarid St^le Paragraphing^ wiJ! be reasonable, with the line of deve- 
lopment easy to follow: The sentence. structure will be isound, though 
perhaps lackirij in vurie^^ rorcerujness. The vbcabulary will 

be adequate, but overall revealing a less sure command of ianguage 
than in the first two categories. 

CATEGORY IV (15-18) - PASS GRADE COMPOSITiON 

Cdmpositjpns jn^ will show the lowest acceptable standard 

in the development of subject and in the handling of language. 

Content There will be just enough relevant ideas to meet the demands 
of a pass grade, but their . range arid depth will leave much to be desired. 

Form and Style The sentence structure arid^ only 
a basic level of literacy, i.e., ah ability to communicate without the flow 
of ideas being .yer/m/^/pJmpeded by faults arid infelicities iri expression. 
Aj)art rrpni this basic requirement, the use of language will reveal few 
other positive merits: 

CATEGORY V [W-\4) - COMPOSITION WITH MAJOR DEFECTS 
BRINGING IT BELOW PASS hINE 

Content Ideas will fail to open up or cover topic adequately: There will 
be a failure to say anything of sigriificarice bri chosen subject. 

Form and Style The choice of sentence structures arid vbcabulary will 
be so Jirriited that the writer has difficulty in expressing his ideas clearly 
and easily: 

CATEGORY VI (9 DOWNWARDS) - OUTRIGHT FAILURE 

Cbriterit Cbmpiete failure to get tb grips with the subject OR a hope- 
iessly^ short composition. 



Fbriri and Style Faults in expression will impede communication so 
seriously that there is a complete breakdovvn in the flow of the compbsitibn. 
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SUGGESTED CATEGORIES FOR COMPOSITION WRITING (SCRE) 



VERY GOOD WORK ' 
Marks '30 2], 26 23, 22 



•...M.any..fe._objert^ 
(iepli)..Aring.in.a uni^ Apparent personal 
involvement in the sabjeci matter. 



• 'Aulhorsliip': involving (I) awareness of the reader 
on the writer's part and appropriate tone vls^a-vis the 
reader, (2) Confidence of the writer in the value of his 
writirij. (j) A 'distancing' of the witer from his 
cxpcricnGe, mater explicitness, the creatioi^ of a 
context for c|nts ani feelings, 3) An orderly and 
selective structuring of the writing, i}] Individualised 
personalised iise of lanpge, involving broad and apt 
vocabulary aiid variety of appropriate si 



• Cprrcctncss in nearly every instance (Spelling, . 
Punctuation and Grammar). 



AVERAGE WORK 
■19,18 — -15,14—- 



• ReasDnableiamher_of ideas, 
oJmliDns,.perhap!L.; 
indicatiniaiaciofical. 



prseeminj pedestrian, 

'hfiaked'jxlvery derivative, 
though appropriate for the 
topic. 



(I) Some attempts to write 
ip^aninpcVthough 
not sustained, or not p'tc 
pro^rlj; filched In tone. 
(2j A willinps to express 
oneseli^^butasenseof 
conviction is lacking in the 
writing. (3j Use made in the 
writing of personal - 
experience or flights of 
iiiiaginatibii; attempts to set 
lliese in contexts, but not 
' riiUyMful;sbme,l)ut 
jidtmiich silccess in relating 
'personal fecliiigs or 
eprienc^tothelarpwbrid. 



nee: 



sentence stnictures, but lacking 
aj)ioluleiipp[Opriatenc$s, 
range and variety. 



•QeMraiiycorrcctjpejiir! 
Grpmarj.Mwtus_eof]all 



with some faults otherwise. 



POOR WORK 



10,9- 



0 



• Few: ideis expressed iitid 



of jsrjnterest in the siibjcci 
shown. ' ■ 



• Npscnscofludieiicc'^ 
struciurc;contex|.iri.cxplicit - 
iulthe.ci>ind,flpwofa 

fcw^thoiights,Limid 

yocabularyandmonptonoiiS' 
simple sentence structures. 



.•Man^.lcchiiicaLfaDliscausiflg 
llie.kwof.ciprcssionlo.bc ; 
im4)ed.ed.{e.g.Ci)raplele.or.... 
aIiDQst.5onipMfailureto use 
full stops correctly). 
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READING 

Three questions, each to receive an impression niark out ofiO: 
Marking should take account of the rolldwing: 

i: the presence or absence of evidence that the text has been read and 
uhderstodd; ^ 

2. the jlresen_ce_ or abs^^ of the text, some emotive 

and/or intellectual and/or moral response to it; 

3: relevance of the answer to the question set: 

Notes: I. Pupils should not be penalised for failing to use tiie approaches 
^^SS'^P^^^^^ so long as they do answer both parts 

of the main question. 

Questions suggesting, as opposed to requiring, Certain approaches 
are: 1 3, II I, II 3. 

2. Answers showing as much awareness of the writer's communication 
and techniques4ind as much maturity of judgment as can r^^dnably 
be expected of intelligent O-grade candidates should receive full 
marks: " - 

A bare pass mark should be given when the answer shp^^^ adequate 
knowledge of the main events of the story and includes some sensible 
response to Jhe second requirement of the questibri, without indica- 
ting that the pupil has reacted to the less obvious qualities of the 
text: . 

Answers which merely 'su story\ rib matter how bril- 

liantly, cannot score more than 4i/lG. 

— _ - - V" 1 , ^ 

3. The followingjpoints should be noted about particular questions: 

r 3: Afow:/?c/70w book required, 

If fiction is used, score oii^^ 

Note that 'features' in the question (a) is plural, but (b)idoes 
not necessariiy mean 'techniques' or 'style' or 'manner bfjpre- 
sentatiori': a good answer to this question could b^ solely 
concerned with content. 

ll 2: Two poems must be discussed for a good mark. 

TWs is <quite a difficu^^^ markers should be fairly 

generous in their assessment of answers to the second part of 
uie qiiestibri. 

''f i • >^ •y^*^''^ di^ly attempts to 

dea! with a whole play will probably be self-penalising): A 
hard question, since not much is done albn^its lines. Markers 
should be fairly generous in assessing responses toilie second 
part, but shoutd expect several sensible comments for a 'pass' 
answer: it should not be an 'easy option'. 

III2rGharacters cjiosen actually be in conflict, 

though this may be iriterpreted fairJy widely, to include, e.g., 
'having widely differing views on the same subject': they do ; 
not actually have to come to blows: 

. ■ Mb . 



SCEEB/SGRE O-grade English Project 
^ 'Criterion Test' 



PAPER II 

Interpretatldn and Language - 1 hour 35 minutes 

iNiSTRUeTidNS Read each passage carefully ^and answer the questions in 
your own words as far as possible: 



PASS/^EA ^ 
J Tbe Return of Dill 

(Scout lis becoming more and more exasperated with her brother Jem's 
attitude to her . . . .) . ^ 

His maddOTing superiority was rabearable^ these days. He didn't want 
to do anything but read and go off b^^ hLmself _StiII, everything he read he 
passed aloni to me, ibut with this difference: formerly, because he thought 
I d Jike it ; now, for my edification and instructiba. " 
^ " *J^^-crawIinl-h^^^^^ _ - 

'Now 1 mean it. Scout, you antagonize Aunty arid I'll ~ I'll spank ypu.' 
With that, I wa^^ morphodite, f H kiii you!' He was 

sitting on the bed, and it was ea^ to grab his front hair aridjMid_one_OH 
his mouthy He slapped me and l tried another left, but a punch in the 
10 stomach sent me sprawling on the floor: It nearly knocked the bieath out 
of rae, but it didn't matter because fknew he was fighting, he wis fiehtine 
. me back. We were sti i 
^ 'Ain't so high and mighty now, are you rj_^creamed, sailing in again. 

^fi the bed and I couldn't get a firm stance, so 1 thr^iw myself 
16 J at him as hard as I could, hitting, pulling, pirichiris, g^)ugins. What had 
begun as a fistfight became a brawl. We were stiil struggling when Atticus 
: . separated us. 

'That's all,' he said: 'Both of you ^ to bed right ndW;' 

'Taahi' I said at Jem. He was being sent to bed at my bedtime. 
26 'Who started it?' asked Atticus Jri resignation. 

^ '.FeiHidid. He was tiyin' to tell mc what to do: I don't have to mind^/w 
now^ do i?' : _ . ^ 

Atticus smiled^ 'Lej^^^^^ it at this: you mind Jem whenever he can 
make you. Fair enough?' 
25 ^ Aunt Alexandra w^ and when she wsnt down the 

h^ I with Atticus we heard her say, ' : . . just one of the things I've been 
telhng you about,' a phrase_that united us again. 

..r^r-^ were adjoining rooms; as I shut the doof between them Jem said, 
- 'Night, Scout.' ^ 

30 _^ _'_Night' I_m_unnured; picking my way across the room to turn dri the 
light: As I passed the bed I ste^pped on something warm, resilient, and 
rather smooth. It was ncHt quite like hard rubber, and I had the sensation 
that It was alive: I also Jieard it move. 

1. ?, switched dri the li^t arid looked at the floor by the bed: Whatever I 
35 had stepped on was gone: i tapped on Jem's dddr. 

'What,' he said. . - 

/Hp:w does a snake feel?' * 

'Sort of rr>ugh. Cold. Dusty. Why?' 

'I thirik there's one under my bed: Can you come look?' 
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40 *Are you beiii' rUhriy?' Jem bpehed the door He was in his pyjama 
bottoms. I noticed not without satisfaction that the mark of my knuckles 
was still on his m he saw I meant what 1 said, he said: *Ff you 

think I'm: gonna put my face down to ii snake you've got another think 
cdmm\ Hold on a mjn^^^ _ .__z ______ 

45 He Went to the kitchen and fetched the broom: *Yon better get up on 
the tied, ^ he said. - : 

'You reckon it's really one?' I asked. This was an occasion. Our houses 
had no cellars; they were built on stone blocks a few feet aboye the groUridj 
and the entry of reptiles was not unknown but was not commonplace. Miss 

50 Rachel Haverford's excuse for a glass of neat whisky every morning was 
that she never got over the fright of finding a rattler coiled in her bedroom 
closet, on her washing, when she went to hang up her nightdress. 
_ Jem madcja tentaUye^ the foot to see 

__ if a^nake would come out: None did: Jem made a deeper swipe. 

55 *Po Ariakes^grunt?' ^ : i . 

*ft ain't a snake,' Jem said, 'tt's somebody.' 

S udden ly a fi I thy brown package sho t Troiti unjler the ibed^ J em rai sed 
the broom and missed Dill's head by an inch when it appeared: 
'God Almjghty.' Jem's voice was reverent. 
60 We watched Dill emerge by degrees. He was a tight fit. He stood up and 
eased his shoulders, turned his feet an their ankle sockets, rubbed the back 
of his heck. Hisicircujatton restor^^ he saidj 'Hey.' 
Jem petitioned Gad again .4 was-speechless: 
'fcouLto perish,'_ sajd Di^^^ 
65 In a dream, i went to the kitchen: i brought faim back some miik and 
half a pan ef cori^^ supper. Dill devoured it, chewing 

with his front teeth, a^was his custom. 

I finally found my voice. 'How'd ydU get here?' 

By an involved route. Refreshed by food, Dill recited this tale: having 
70 been bound in chains and left to die in the basement (there were basements 
in MeridiiMi) by his hew father, who jdisliked hiiri^ arid secretly kept alive 
on raw field peas by a passing farmer who Jieard his cries for help (the 
gppd man pocked a |fu&hel_ pod by jJod^thrpugh tJw yeritilato^^^ worked 
-- himself free by pnlling the chains from the wail. Still in wrist manacles, he 
75 w^dered. two nnl^s out of Meridian where he di^^^ 

show and was immediately engaged to wash the Camel. He travelled with 
the show all dyer Mississippi Unti^^ 

he was in Abbott County, Alabama, just across the river from Maycomb. 
He walked the rest of the way. 

80 *How'd you get here?' asked Jerii. : 

He had taken thirteen dollars fj:om Jiis mother's purse, caught the nine 
Pl'clolckTrpm Meridian and J^ot off at^ He had vvalked 

ten or eleven of the fourteen miles to Maycomb^ off the highway in the 
scrub bushes lest the aU th qri ties_ be_ seeking^ him, an d had ri dderi the_r(^ 

85 mainder of the way clinging to the backboard of a cotton w-agon. Me had 
been Under the bed for two hours, he thought; he had heard Us in the 
dinirig-rbom, and the clink of forks on plates nearly drove hint crazy. He 
thought Jem and I would never go to bed; he had 'considered emerging 
arid helpirig me beat Jem, as Jem had growri far tdller, but he knbw Mr 

90 Finch would break it up soon, so he thought it. best to stay where he was. 
He was worn but, dirty beybrid belief, arid hbriie. 

You should use your own words, as far as possible, in answering the foilowing 
questibris: 
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1. id) Explain the change in Jem's relationship with his sister which is 

referred to in the first paragraph. 

(6) Why was Scout pleased that Jem fought back when she punched 

him? (3^ 

2, ' "Taah!" 1 said at Jem. He was being sent to bed at my bedtime.' 
(line 19) 

(a) How does Scout feel towards Jem at this moment? 

(Bj Explain how yon Joiow this from the two sentences quoted. {4jf 



3. (ja) What was Scout's usual attitude to her brother? 

{b) How do you know this? (4) 

■ • 

_ ' . ' V 

4. (a) What was Jem and Scout's reaction to finding Dill under the bed? 

(b) Why did they react as they did? 

(Consider that Scout was 'speechless' and moved *in a dream', and 

that Jem's voice was 'reverent' when he s,aid 'God Almighty'.) (4) 

5. Two accounts are given of how Di'l had returned from Meridian to 
Maycomb. (lines 69-85) 

(a) Why tides Jem repeat Scout's question, 'Hbw'd you get l^ere?' ? (1) 

(^) What was the real _r^^^ 

(Answer asTuiiy as you can, considering Dili's relationship with 

Jem arid Scout.) ^ (3) 

(c) Judgingfrpn* t 5 way Dill first tells the story, what kind ofcpeibOn 
would you Sc, he is? (3) 

6. What cdritrast is therebetween family relationships in Jem and Scout's 
" " ' and in Dill's family? (3) 

25 
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PASSAGE B _ _ ^ 

The Excavation of Ur 

Then Wooiley made his most sinister find: the Royal graves of Ur con- 
tained the rjemaiiis of commoners as well as of royalty. 

In one tomb lay a number of soldiers of the guards wearing^ copper 
helmets arid with spears in their bony hands: At the farther end of the 

5 chamber lay nine ladies of the court, still wearing, the elabpra^^^^ 

headdresses that they must have donned for the funeml ceremony. By 
the entrance stood two heayj^px-drwri charts; i^^ the drivers' 

bories, and at the^ oxen's heads lay the bones of the grooms. 

^ fn the jrave of Queen Shul>ad ladies of the cour^ were found lying in 

10 two parallei rows^: At the end of one of these rows was a man's skeleton ~ 
that of the cpurtjiarpjst^. Jlis^arm were still lying across his broken 

instrument, which was ornamented with a calf's head in lapis lazuli and 
gold. Apparently he Jiad held fast to his instrument even as death overcame 
him. At the wooden bier where the Queen herself reposed, two female 

15 skeletons were fotind in a crouching position. 

What did ail this mean? : ___r._ _ : : :: :: i n i : l 

There was only one explanation : here the greatest possible sacrifice had 
been exacted of mortal rheh - theiF<»wh lives. Wbolley had stumbled oil a 
scene of planned human sacrifice, carried out in conformity with the king- 

2b god principle. The ppsitiori of the skeletons, as ^^^^ other drcUmstailces 
of the find, indicated ihat the victims - courtfolk, soldiers, and servants - 
had died quite peaceflilly arid it is thought probable that they_ walked to 
their places, took some kind of drug, and lay down ; after the drufe had 
worked, whether it produced sleep or death, the 4ast touches w^re given 

25 to their bodies and «arth was flung in and trarripled down on the top of 
therri • _ _ _ 

What conclusions did Wooiley draw from these finds? 'In rib^ k^^ 
text,! he writes, 'is there anything that hints at human sacrifice of this sort, 
nor had archaeology discovered ariy-trace of such a custbril br any survival 

30 of it in a later age; if, as I have suggested above. It is to be explained by 
the deification of: thAeaHy i^irig^^^ say that in the historic period 

even the greater gods demanded no such rite: its disappearance may be 
an argument for the high antiquity of the Ur graves:' i - 

C. W. Ceram. 

Use your own ^rds as far as pbssible in answering the following questions: 



1. (a) What was it that Wbplley fbUnd 'sinister' about the graves at Ur? (1) 
(h) Twp things in particular abbut the gravies required explanation. 

What were they? ^ i^) 

2. What explanation for the find is giveri iri Paragraph 5j? (2) » 
3; What belief made this practice acceptable to the people of Ur? (1) 

4. Accbrdjrig tb the last paragraph, which two facts led WoolIey to *L 
conclude that this burial place was exceptibrially bid? (2) 

5, (a) Why is a colon (:) used jri hrie I? (i) 
ih) Why is there a semi-cdlon (;) instead of a full stop iri hrie 7? (1) 
tc) ' . * . the victims - courtfolk, soldiers and set-vants - had died 

quite peacejujly ... * (lines 21-22). Explain why the dashes have 
been used here: ' , 



ERIC 



6. (a) Why does the first sentence stand as a separate paragraph? (1) 

• (b) Why is the question, 'What did all thrs mean?' also written as a 

; separate paragraph? I (2) 

(c) Explain why the last two paragraphs are separated From each 

other: : , (1) 

7. Write a summary of the passage down to 'trampled down on top of 
them': 

bo not inciade the last paragraph in your sunimary. 
The following plan may help in writing the summary: 

(1) What was found in the tombs at Ur. 

(2) The significance of the find: what it showed about royal burials 
there. 

Your final summary should be in one continuous paragraph of about 

100 of your own words. (10) 
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SeftE/SeEEB 'CRitERieN TEST' 
' PAPER II 
Marking Scheme 



PASSAGE A. 
1. * 2 marks for {a). 
1 mark for (b). 

2 marks for id es^^^^ aspects of Jem's behavipur 

haye chanj^d^OR for giving two reusons why he has changed, 

OR for giving one of each. 



ASPECTS OF BEHAVIOUR: Any two of: 

1 . He rib longer played with her I) OR he stayed by hiriiself (1) 
OR is isolated (I) OR reads a lot 0): 

2. He treated her as an inferior (I) (was superior^V) OR he 
treated her as a child (I) OR behaved as though he was her 
father or an adult (1). 

3. He tried to teach her things (I ) OR educate her (1) OR give 
her instruction (A). 

'passed on things for her edification and instruction' =0. 
'bullied her'=0. 

REASONS FOR eHANGE: 

I : He was growing up (I) OR felt adult (1) OR felt responsible j 
(1) OR important (1) (superior^ j 
2. He was no loriger interested in her [\). . \ (2) 



(b) It proved that they were still oil the same level (1) OR that he 

^ wasn't really superibr^ti)- . \ ' 

(eqnal or eqaals=i unless expanded - e.g., equal like two ' 
. children in a fight=l) . . (1)^ 
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2 marks for (a). 

2 marks for (b), 

) ..... .... ... 

(dj Give two marJcs Cox clear statement of one of the following ideas: 
Triumphant (2) OR crowing (2) OR pleased at his humiliation 
(2J OR: pleased that he's being taken down a peg (2) OR 
: mocking (2). 

~ SGive one.'mark if the above ideas are absent but any of the 
' following are present: _l 

' Mnctii*. \ "Ot feeling friendly towards him (1) r\T> e^;* r„i rw 
"^'^^'^ Indt at one with him (1) ^P'^^^"" 

' trying to^et at him, arinby him (^1 ) _OR_Seased that he's 
made equal with her again (1) (without any implication that he^ 
has been re<fMce<f td_her leveO^ _ 
Angry =0; annoyed =0; sarcastic =0. 

(N B. For two marks; her sense of triwnpii or mockery must 
be. noted.) v 



(6) One mark each- for my two of the following: 

An explanation that Ta^ is an expression of derision or 
of triumph (1) (sarcasm=d); . 
'She sounds mocking or triumphant'^ 1, if spedficdlly 
linked to Taahl\ 

2. An explanation that 'at' has been used intead of 'to' (to 
convey antagonism) (1). 

3. A cordiherit on -sent to bed'i showing that this humiliates 
Jem (1); e.g., 'he is sent to bed like a child'=l. 

4. A comment that Jem is humiliated by going to bed at d 
child's bedUme (1) or at his younger sister's bedtime (1). 

(Note: An 3 and 4 some awareness that it is humiliating for Jem 
must be conveyed. 

e'g,^ 1. know it because he had to go to bed at her 
bedtime'=e.) 



2 for (a), 2 for (5). ^ 

(a) Any two of the following: . " [. ,. 
• 1. Trust (i) OR reliance for help (I) (or for protection (.1)): 

'2. Friendly (1) OR e.g.; 'they got on well' (1) (united=|). : 
3. She regarded him as an ally (1) OR e.g., 'they were on the 

same side' (I). 
If (a) is wrong, give 0 for (6). 

(b) Answers to Jib) must be related to answers to (a) 
Any two of the following: 

1. She sought his help with the 'snake' (1). 
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2; Either reference to the friendly exchange of 'Good night' (1) 
OR reference to 'united lis again'. 

3. Referenee to th^ factth^^^ were united otgainst Aunt 
Aiexandra in some sense (I). 

N.B. Quotations are acceptable answers in (fi): 



2fbrja), 2 for (6). 

W QAod an^weriishpuld^^^^^ only Ihi^ surprise of Jem ahd 

Scout but aisD the intensity of their amazement, the sense of 
awe or unredtity >(^hich seemed to come over them. So : 
Surprised=l [ y^iy^ s^^ or astbnished = l ; 

absoiateiy astonished =2; couldn't -believe their £yes=2; 
shbcked=2j^stunned=2i awestruck=2. ^* 
If id) is wrong, give 0 for (6). 



ip) - Qric mark each for any 2 of the following: 

I. They didn't expect him (1) OR they thought he was in 
Meridian (1): 

2- The ptdce in which he was found was Unlikely (I) (e^., 'they 
didn't expect to find anyone under the bed', OR they were 
expecting a snake, not Dill. 

3. The situation was so tin usual as to seem unreal (1). 

4. They realised that there was_ a 5er/oM5n^^ in Dill's presence 
there (i) OR that Dill jnust have had a very good reason 
for being Jhere i IX PR_ that this vya^^^ a _situation _\vhLch 
would be hard to explain (1) OR which could cause some 
trouble (I). 



(a) He doesn't believe the first version (I). _ 
{h) 3 marks: 

1 . lis for picking up the very last phrase of the passage 'and 
. home' e.gv'D>ll f^elt that Maycomb^was his real home'^ I ; 

OR 'he used to live in Maycbmb'=l. 

2. The other 2 are for a gloss on 'home' - some explanation of 
what 'and home' implies. Give one mark for each of any 
two of the following: 

~ Dill felt that Jem and Scout were like his brotherand 
sister (I). 

- bill felt at^ase with Jem and Scout (I) OR happy with 
them (1) OR wanted to be with them (I)- 

- he was unhaf^y in Meridian (1) (but iibt 'his father ill- 
treated him')- 

- he trusted Jem and Scout (1) OR felt secure {\y, with 
them. 



s 
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- he felt the>^_were cbncerried (I) OR interested in him (ij 
(OR^interested 6^ him (I)). 

- he missed Maycomb (1) OR he missed Jem and Scout 
(1) OR he was veiy fond of them (1) (liked them=6). 

(3) 



Personal qualities/traits required, not just his situation, 
(c) 3 marks : Aiy 3 of : 

1. Imaginative (!) OR fantasiser (1) OR dreamer (I). * 

2. Story-teller (2) OR romancer (2) (this includes 'imaginative'). 

. 3. Liked to be the centre of attention (1) or liked to cause 
amazement (I). 

4. Was really unhappy, but fictionalised it (1) or had poof 
relations with his father but fictionalised the fact (1) (hot 
just 'had bad relationship with his father'). 

5. Had a sense of humour (1). 

m 

" Liaf^O, unless it is clear that the pupil means only '^s a fantasist' 
or 'for amusement' or 'as a way of coming to terms with un- 
happiness*. 



6. Give an impression mark out of 3 (half marks allowed). 
Points to be cbrisidered : 

Dill's family: 1. Dill _has poor relatibriship with his 

parents. 

Jem and Scout's family: 2. Atticus diisprayis understanding of Jerri 

and;Scout. 

3. Jem and Scout have basically a happy 
relationship. 

4. There is a mutual bond in Jem arid 
Scout's family^ 

5. despite surface disagreement. 

Three marks for a convincing statement otcontrast between the two 
families iri terns p^^ relatibriships (riot, e.g., their social 

standing or their money). 
If rib cbritrast=0. . ^ 

- ^ (3) 

25 

PASSAGE B. 

1. (s) One mark for the likelihood that the commdriers had beeri put 
to death (1). Accept 'the fact that there were ordinary people 
there as well as royalty' = I . 

(No penalty for 'lifting' the word 'cdmriibriers'.) (1) 

{h) One mark for each of two points : one pdirit abdUt the presence 
bf ordinary pebple (as well as kings and queens), and one point 
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about the conditions or circumstances in which the bodies were 
found. 

Penaliise lifts ($). 
So: 



! . EITHER - whj5^ were there brdiriiuy I 
OR why were so many buried in the same piace?= i 
OR why had the ordinary people died?=I. 

2. EITHER - a generalised reference to XYi^ positions of the 
bpdies=l, e.g., 'why were they lying in peaceful or 
natural positions?^=l OR 'why were they lying in the 
positions they were iri?' = 1 . 

OR a generalised reference to the formality of their dress^ U 
e.g., 'why were they dressed so elabdrately or richly?' = I 

OR a generalised referrace^t^^ found there_==l, 

e.g., 'why had they been buried with the things they 
normally worked with, such as spears, carts and musical 
instruments?'=l. - 

Particular examples=|, e;g., 'why were they wearing head- 
dresses?'=i OR 'why was the harpist still holding his harp?'= 
i etc. 

2. One mark each for any two of the following: 

1. they_had been /?mMc> death (or died) when the king or qiieeri was 
buried =1 (had been sacriRced=^, it was a human sacrifice = j). 
Must show understanding of 'sacrifice'. 

2. they had^ied mllin^ly (=1) OR calMy (=1) OR without a 
- j/r^gg/ff (== 1) (peacefully=i). 

as matter of course = 1 ; suicide^ I . 

They had (willingly) sacrificed themsel ves when the king died=2, 
(Accept 'sacrificed' if in the phrase 'sacrificed themselves'.) Must 
show understanding of 'sacrifice'. 

3. because the king was a god (1) OR because the king needed his 
followers of servants after de^th {=^\) OR because they were 
ready to join the king in the next Iife=l 



3. They believed that the king was a god= i OR They believed in life 
after death = I . 

King-god=i. 

4. One mark for each of any two points : 

1 . There are no written records of the custom 1 
OR it is not recorded in history— \ 

OR since history began no kings or gods have rqquired human 
ice==F £1n the historic period'^}: extensive lifts to be 
"^ly penalised): 




Archaeologists have never fouad any sign of the custom ui any 
^<^i^rJPl^^—}X^^^^ Pj" in jmy other place= I . 

(Archaeologists have never found any sign of it^ l). 
No evidence in later tirties^^ 1 . 
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(2) 



(2) 



(1) 



(2) 



5. (a) One mark for an answer which indicates awareness that the 

second part of the sentence is an exptonaiToyi of the first part. 
E.g., the colon is used because the second part explains why it 
was sinister = I , OR the second part states what Wodlley founds 
1. . 

(b) One mayc for an arisv^^ indicatirig awarerie^^ second 
part is an ejcpumion of or elaboraiwn o/i the first part; 

fe-E-A semi-cblo^ becajisc the se^ par^ (or the 

second sentence) goes on to say more dboxxi the carts— j, OR 
"... is still refemrig to the carts'^ 1, OR ' . . . tells Us more 
about the carts' ^ I . 

(1) 

\c) One mark for an answer indicating awareness that the dashes 
enclose a parenthesis: 

E.g., they separate the words from the rest of the sentence^!. 

OR the sentence would make sense without the words between 
thedashes=I. , 

t h ey a re u sed to al l b w the w ri te r t o explai ri who the vi c ti m s 
were without starting a separate sentence^ 1 . 

Most show awareness of idea of parenthesis. 

• _ (1) 

6. (a) One mark for EITHER 

it is an inlroductory statement^ J 
OR it states what the passage is going to be about - j 
OR it is a gm^vai statement, before the other paragraphs discass the 

details of the firid= 1 . 

^ ») 

f ^) Two m a rk s : one fo r a wa ren e ss th at , in a sen se, t h e question 
sums up the preceding paragraphs, or indicates the end of a 
section ; brie fbr awareness that it introduces the answer. 
So: . 

ft is a turning point in the passage ^2 (2) 

• OR It links the two paragraphs above and below it — 2 

It gathers together the writer's thoughts about what was found - 
1 , and it leads on to his explanation of the find^ 1 . 

(c) Differenj topics=l. One is explanation of the meaning of the 

find, brie gives Wbolley's coriclusioris about it — I. (1) 

7: SUMMARY 

The summary should be given an inipression mark out of 1 0^ taking 
into account the presence or absence of the points listed below, fluericy, 
linking words and lifts from the passage: — 

(10) 

Muin points: 

Lines 1-15 The presence of ordinary people, soldiers, courtiers and 
servants, in the royal tombs. 

The type bf objects they had with them, and the dress they 
were wearing. 

The naturalness of their positions. 
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Lines 16-25 . The circumstances Indicate that these people had been 
killed, ^ 

because the king was regarded as divine. 

They had died calmly, accepting their fate, 
. probably after taking a drug. 
. They were then buried^ 

Undue Length 
Penalties are as follows: 



Stimmary of more than 110 words: mark out of 9 
Summary of more than 120 words: mark but of 8 
Summary of more than 130 words: mark out of 7 
Summary of more than 140 words: mark out of 6 
Summary of more than 150 words: mark but bf 5 
Summary bf mbre than 160 wbrds: mark but bf 4 
Summary bf more than 170 words: mark but of 3 
Summary bf thore than 180 words: mark but of 0 
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SCEEB/SCRE O-grade English Project 
'Criterion Test' 



PAPER III 

Tactual' and ^Persuasive' Writing - 1 hoiir 



INSTRUCTIONS Answer TWO questibhs. orie fr^ each section. You should 
spend about 30 minutes on each question: 



A — 'Factuar Writing 



Do EITHER I OR 2 W 

!. The Essentials of Jumping' might be the title of an article on shb^jumpirig. 
v\;guld pr6bab!i_have ashort intrbductiori to catch the interest of a reader; 
the bulk of the article would instruct a beginner in the object of the sport 
arid in the most important skills involved. 

: CbWse a sport or pastime that you know well, and write a short article 
on it of about 200 words suitable for inclusion in a handbook of sports 
arid pastimes. 

> . The title might be The Essentials of ....... \ 

OR 

2. Explain cj early and logically, as though to help a friend who didn't know 
how to do it, what is involved in one of the following: 

(or) washing and setting or blow-drying hair. 

(b) cooking a stew, with potatoes arid vegetables of your choice. 

(c) rbutirie care of a dog, cat or other pet. 

(d) playing^. draughts OR dominoes OR ludo OR snakes and ladders OR 
a common card game. 



B — 'Persuasive' Writing 



Do EITHER 1 OR 2 OR 3. 

1 : In th e sch odl m agazin e, . th ere are to be regbrts on the actiy i ti es af varij>u s 
schobl societies. A society you belong to needs more members; to attract 
them,' you are asked to give the sort of report which will interest people 
irijbiriirig. 

Write your report. 

OR 

2. The Headteacher of your school has asked for a statement of the senior 
pupils' views on the jdea of a School Cotmcil, cdnsistirtg of elected staff 
arid pupils, tb deal with schbbi rules and major breaches of them. 

: Wr\i&, your statement, arguing as persuasively as you can for or against 
the idea: 



OR 



Write aii article for a school magazine arguing for or against ON^ of the 
following: 

. (a) Compulsory sports at school. 

(b) Cookery or Housekeeping for boys: 

{c) Technical Subjects for girls, 

(df) Reducing the school leaving age iO 14. 
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Tactual' and 'Persuasive' Writing — Criteria 



Marks jO - 



■2r 



_ Tacluali : j: ^ 

% ThoroughJknowledge of 

subject sho>4ai. Good ^ 

je/pc/Mrof jTiost impoctarit 
Ppints CespeciaJly in No. /). 
Al jjdemands of the i 
question met. ^ 

Per sags ive_ 

A defin i te ppsi tip n fi rmiy 
stated. 5eyerfl_/ reaspnable 
arguments in support. 

I Good Ibgical or 
chronological organisation 
of material. Clear 
paragraphing where 
aj^rbpriate. 

I Awareness of the reader. 
Appropriate tone aiid style 
for the *audiehce* and 
fuhctidh defined by the 
quesCidh. 

I In *Tacruai\ untramfnelled 
expression, accurate . 
vocabulary, clear sentence 
structures :Jii :*^^W5/flj/ ve*, 
axange btappropriate 
vocabuJary^pQssibly fairly 
successfuLi^_of_emoiive 
JaDgu_age anoTheiorical 
sentence structures. 

> Cprrectness in nearly every 

ms tance (SpelH ng,^ 

Punctuation and Grammar). 



AVERAGE WORK 
:Faciual 

\ Knowledge of subject. 
Some attempt at seteciion 
of jnain -points, bat a 
tcndency toJry to _L 
include_everything or a 
tendency ta omit 

important points. 

: Qdbasjojnal irrelevant 
points made. 

Persuasive 

Ppsi tion is stated. 
Argunients ipiupppriL 
fewer in num^r and less 
forceful than in good 
answers. 

• Some attemp t at 
brganisatibh bf material, 
. but hot fully siiccessful. 

• Paragi^phihg where 
appropriate. 

9 Attempt to write for the 
sp^ified-*audicnce',-but hot 
.sustained, or not quite 
properly pitched in tone. 

• ln\Tacmal\ fairly 
straightforward clear : 
vi>cabulary:and sentence 
stnictures,4jerhaps with 
tendencyLtOTamble or with 
some vagueness. In __ 

* Persuasive** adequate _ 
vpcabuJaryaLnd sentence 
structures ; ppssibly 
attempts at emptjve 
language and rhetorical 
devices. 

• Gerierall]^ correct Si>el ling, 
Puhctuatibn and Grammar; 
correct use of full stops 
almost always thbugh with 
sbme faults. 



POOR WORK 

- Tacumt ^ 
i tittle real knowledge of the 

subject rhown: No attempt 

to select main points: 

Qniission of many . . 

important points: Much 

may be irrelevant. 

I NQ_atteoapt at or^nisation 
of material.-ahotchpotch 
of ideas/points. 

> Np awareness shown of v 
*audience\ 

% Failure to exhibit cpmrnapd 
of adequate ypcabulary and 
sentence structures. 

ji Many technical faults, 
causing the Rbw bf 
cbmmunicatioh tb be 
impeded - e.g. cbmplete or 
almost complete failure tb 
use full stops cbrrectly. 



SGEEB/SERE O-grade English Project 
'Criterion Test' 



PAPER IV 
Ihterpr^tatibh-Multiple Choice 



PASSAGE BOOKLET 

IMSTRUCTIONS Do not write anything^^^^^ this i?bbJklet. Read each Passage 
carerully. Then answer the questions by writing the 
LETTER ofjhe correct answer iieXt to the question 
-w number on the answer sheet. 

PASSAGE I 

FPt J^l^scyute effrontery in^ selling fake anticiiies i have yet to come across 
anyone to beat Herbeii Pomfret: Gentie-mannered and charming, with the 
slightly pathetic air of an aristocrat w^^ timei^ and is 

down to his last Rolls-Royce, Herbert gives each prospective victim a talk 
L^S on the perils of investing in antiques nowadays and the Svicked tricks some 
unscrupulous villains play on people who give them: their trust. It is a 
technique which rarely fails. _ 
Herbert finds his victims by inserting small-ads in Personal Columns. 
- Usually he offers a single item^ a superb Louis XVI table, perhaps; a 
10 RenpLiia iiarnond and ruby ne^^^^ is selling a pairiting, he quickly 

dismisses the victim's attempts to talic money: 'Before I could even think 
Pf J^jscussirig a price,' says Herbert, 'I want you to consult ah irideperiderit 
expert.' 

He in si sts on the vie ti rii removing the pic tu re fro ill the waj 1 and tak in^ 

15 at, in a hired c^r,^ to any one of the art galleries in the Bond Street area. 
There, while Herbert waits in the car at the parking meter, an art expert 
examines the picture in return for a spot consultation fee and declares the 
canvas to be genuine. He might say: T estimate its value at approximately 
£3,000.' , : - , 

20 -The victim returns lo tiie waiting car with tiicpicture and offers Herbert 
^2,000. Whereuj5pn Her^^ his stuffi I'll^ive him that), sighs, 

'I was expecting the value to be nearer £3,666. Ah^ well;' 

He accepts the offer, art^ PD receiving tiie victim's ofieque ma^ 
reasonable request, 'it will take me three days to find a suitable replacement 

25 to fill that gap oil my wall,' he says. 'Also, it will allow tiitie for your 
cheque to be cleared. To prove that the sale has been completed, and that 
the painting is yours, please be good enough to sign your name on the 
back pf the canvas herei N^Mth^^^^ : 
The victim signs. Three days later he collects the canvas from a distinctly 

30 col d iM r Ppmfret wh b points o u t that he has _<1 i scp vered the paint ing to be 
indeed worth £3^066; nevertheless, he is prepared to honour his. bargain: 
He hands over the c_ariyas. The victiin's signature is on the back. Hp^^^ 
goes the victim with his purchase, a little regretful, possibly, at having won 
a deal with a defenceless elderly gentleman, but elated at having made, he 

35 believes, £1,660 profit. 

Probably not until years later, when he comes to sell his precious in- 
vestment, dof^s he discover that the painting is a forgery. Herbert had con- 
cealed it in the back of the genuine picture's frame. It was the forgery the 
victim signpu. : l '_ 

46 At least he can'r say that Herbert didn't warn him : . : 
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PASSAGH 2 

tAfter many yeur*; - ile is iiHowed to visit her husband who is in a prison 
campj 

They had only half an hour, and the seconds passed quickly, like grains 
of sand trickling through the neck of 'an hour-glass. Dozens of urgent 
questioHS crowded Natasha's mind, but all she asked was: 
*When did you find, out ubbUt the visit?' 
*P^y J>erore yesterday. And you?' 
'Tuesday:' 

are things at wpjrk?' he enquired. 
*Why do you ask?' she said anxiously, 'bo ydU know?' 
*What do yoU mean?' 
10 He knew certain things, but he wasn't suri; what was iia her min 

He knew for instance, that the wives of prisoners were always being 
harassed. But iiow could he know that last Wednesday, his wife had lost 
her job because she was married to him? Having received only three days 
ago a rioUce granting permission for this visit, she had not yet begun to 
15 look for new work: She had waited for their meeting, as though by some 
"I't^clc it might-^iye ijlumiriation, showing her what to do: 

But how could he advise her? He had been in prison for tod long and 
knew nothing of the \v<H^^ • 
Thcjgreat decision she had to jnake was whether to divorce him or not. 
20 ^ A s the minutes slipped by in thjs drab, poorly heated room with its dim 
light from the barred window, her hopes of a miraclc were fading. 

She realised that [n this meagre half hour she could not give her husband 
anj^ idea pf her loiieliness and suffering, and that he had a life of his own, 
which had taken a quite different course from hers. So why should she 
25 upset him wjth her worries, \^ " ^ ^ ;<:T7t notiiing to him anywey? 
The warder turned to tl: • ■ ;n to riudy th< plaster^ 

;Are you sM allALone? A .* ?^ he asked still tenderly rubbing 

his cheek against her hand: 
^Was she sad?' he askeu! *t v^vS reariy over, she felt ^ibk at 

30 heart. She would soon have /uo tiic bleak streets feelinj riqrie 

the better for this rieei:- a . -;ronc. Her dre^iy enveloped her 

like grey CO ttbn wool. 

'Natasha!' He strokeu her uiinjs. ^If you ecu i. it jp, i haven't much 
rridrc^to go now - only ihrce years. Only (hree . . : ' 
35 *Oniy thfeelV She stopped hint angrily, feelings her voice, tremble^ and 
losing cbntroLc^ it._ ■ Only throe? Only, you say] I'm at the end of my tether. 
1 won Y last another month! I might as well die. The neighbours treat me 
ci'rt - they've throNVT} out rny trunk. I've stopped going to sec my 
sisters and my aunt - they all jeer at me and ^ay they've never heard of 
40 such a fool. They keep telling me Jo divorce, you and remariy. When is 
aH this going to.end? Just look at me! Vm thirty-seven years old. In three 
years ri I be an old woman. 1 come home and I don't make myself dinifier; 
t <fe>rij clean the ropm -- f haven't the heart: f just flop down on thecoUch 
and lie there like a log: 1 beg you my darling, please do something to get 
45 but earlier.'- 

She had not meant to say any of this, but it was alt too much for her. 
Shaking with sob^ai^^ husband's hand, she let her head fall 

against the rough warped httle table, which had seen many sUch tears. 

'Please calm yourself/ the warder said sheepishly, glancing at the open 
50 door. s 

The lieutenant-colonel stood grimly in the doorway, glarinj^ at the 
woman's back^.and shut the door. 

The regulations did not explicitly forbid the shedding of tears, but, if 
one went by the spirit of the law, they clearly could not be permitted. 
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PASSAGE 3 



A gf^t deal of if fespdrisible rioriserise is written about the effects of sex, 
• viplehce and bad Janguage on television. _ 

It is quite right that we should be cdncerned with^the effects of thesci 
but the fact is that detailed analytic research from' Himmelweit to Haiioran 
has failed to produce conclusive evidence to justify the imposition of external 
cerisorship^over teje^^^^ 

BBG and iT>thave detailed codes of practice and have conducted research 
l"^P the effect o^ A considerable percentage 

of advertisements is already rejected and pressure on certain programmes 
i<? quickly effective. i L _ _ _i 

The latest responsibJe contribution to the debate has j^st been made ? 
available by Southern TV, wh carried out an investigation intbjgpbd taste 
on tejevisibri'. The aim w^is to assess the level of feeling among parents 
about sex and violence on British television, but riot, it must be emphasised, 
to Pibve any j^Mse an 

Several hundred interviews were carried out under the difectibri of two 
triiin*^ psychblb^^ was prepared. A distinction 

was drawn in the conclusions betweeii spiontOTeous, unprompted criticism • 

response to gene^^^ and criticism emanating from direct ques- 

tioning on specific points. 

T^Le coriciusioris do show that a cbnsideFable number_(^: parente are 
concerned about the effects of violence on children, but comparativeiy few ^ 
can recall specific, harmful incidents that have affected them br their ^ 
children. 

A sizable body of opinjon expects harmful lasting effects on young people 
ifL general with hardliners ori this in the 45-plus age-group. The main 
objections were fears of imitative behaviour, personal embarrassment at 
watching certain scenes arid a feeling thai televisibn- was encouraging 
passivity. 

Probably the riibst important cbnclusjori^ to emerge is that mbst parents 
consider the existing safeguards for the protection of children adequate. 
There is certainly little evidence of any general drive for the imposition of 
a more exacting censorship. : 
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PASSAGE 1 

1 . WhichLof the follDwing could not be 
regarded asjan antique? 

A An oJd but elegant jchair _ 
B A ninetcenth.century tea-pot 
C A four-posler j?ed 
D An_elcctric clock 
E A Roman coin 

2. Tb.e phrase closes^ in meaning lo 
^absolute eJ*ronteiy* (line 1) is 

A utter dcccitfulness 

B thorough greed 

C downright trickery 

D sheer impudence 

E considerable skill. 

3. In the first paragraph Herbert Pom- 
rrct\wirris prdsprctive buyers against 
unscrupulous dealers because4ie 

A is an honest dealer himself 
B wantiSv/biiyers to avoid hi.s 

cornp|titoi^ 
C wants To win the buyer's 

confidence 
D is interested in the tricks of 

dishonest dealers 
E liki^to talk business with his 

customers. 

4. The_ word .nearest in meaning to 
*patheUc' (line 3) is 

A stupid 

B inadequate 

C pitjM . 

D unhappy 

E hopeless. 

5. Herbert's ^technique which rarely 
fails' (line 7) is to 

A encourage the buying of 

antiques 
E warn a^irist swindlers 
G ^rive a Rolls-Royce 
D advise a^ihst buying Antiques / 
E pretend to be upper class. y 



9. 



6. The expression 'knows: his staff' as 
used In line 21 rneans that Herbert 
A is a good judge of the value of 

antiques : : / . 

B can recognise potential victims 

; easily 

C knows how to trick people 

cleveriy 

D is_^ood at pretending to be 

knowJedgeablp 
E can always obtain antiques 

when he wants them. 

7. Herbert's main reason for wanting 
to keep the picture^ for three days 
longer is to 

A obtain a similar picture to put 

on the wall 
B copy ^he victim's signature on 

to a fake 
C check oh the real value of the 
- picture 

D remove the genuine picture 

from the frame 
E 'insure that the victim can 

actually -pay. 



The phrase *. : a disiincUy cold 
Mr Pomfret : : " (lines 29-30) means 
that Herbert 

A did not care if the buyer was 

: dishonest ft : : 

B spoke Jn a very clear_voice 
G was polite bu t unfriendly. _ 
D was angry at being cJieatod . 
E was_sorry to part with a ^ jofl 
painting. « 

In iines 32-35 the buyer probably 
feels all_ the following except 

A a jittie ashamed 

B proud of his business sense 
C flushed with success 
b sorry for Herbert 
E slightly foolLsh. 



10. All the roUowihg encourage the 
buyer to think he^is clicatihg the bid 
itiari except that Herbert 
A makes him consult an 

independent expert 
B ddeis not accompany hiiti to 

the expert i - -- 

C acccp.ts his Valuation of £2,000 
D honours the barjpin made 

with him 

^ E asks him to leave the picture 
for three days: 



11. Which^x>h& of the follbwieg is dot 
part of Herbert's^rriethod of winning 
tlie^Gtim's cbhfidehce? 
A The fact that th*:' art expert is 

in^ his pay 
B His't^jUingoe^ to have ah 

Outside evaluation 
C His cultured style and courtesy 
D His hiring a^^car to gO to the 

art ^Wety 
B The sense of guilt he creates in 

the buj er 



PASSAGE 2 

12. The opening cbhversatibh between 
Natasha and hor husband cbuld 
best be described as 
A casual 
B angry 
C strained 

D ronnai 
E intimate: 



13. According -to unes 11-16, Natasha 
had ^hoped that her visit to her 
husband would : 

A console her for the loss of 

her job _i • 
B prevent Jier from being 

harassed 

C siigges^t ji_lj ae of actioiL lo lier 
D inform heV Jiusband that she 

had lost her joli : ^ 

E cast, ji^ht on her gloomy 

existence. 

14. Natasha might \yish Jo seek a 
^iVP^ec for ail of the following 
reasons except that she ' 

A had been made a social outcast 
B no longer Ibved her husband 
C had grbwii tired of waiting 
D hbped tb get ahbther jbb 
E was desperately Ibhely. ^ 

15. Which one of the fbllbwihg best 
describes the attitude of Natasha's 
husband during the Visit? 

A Loving but insensitive 

B Bitter and unfeeling 

C Inquisitive but symptitlietic 

D Depressed and pessimistic 

E :\nxious but hopeful 

16. TifiJiusband's queslionsLin_iine_27 
revealed most clearly to Natasha 
his 

A lack i3f understanding of her 
real situatioja. 

B sorrow at being separated 
^ from her 

C complete understanding of her 
unhappincss 

p unselfish nature 

H stupidity in'Asking obvious 
questions. 



17. All bf the follbwihg factors contri- 
bute tb Natasha's unhappiness 
except _ - - - 

A her sense of isblatibri frbm her 
-^husband . 
B heiv relatives' opinion bf her 
C the Unldtidticss of those 
- around her 

D her scorn for her husband's 

lack of spirit 
E- her sense of the poihtlesshess 

of life. 

18. TJbeiharshnffisiof Natasha's situ- 
: atioji is^ sugg^led jn the passage by 

all of ihfefollowijig except : 

A ^he use of adjectives expressing 

Lack^ of CQlo_ur_and hope. : _ 
B her jsoiation from friends and 

relations ■ 

C the cmeity of the warder __ _ 
t> the jact that she has. very little 

time Mth her husband ____ 
E her lack of interest in her home: 

19. Accbrdihg tb the passage^ which 
bhe of the following is true of the 
rbbm in which the visit takes place? 
A The plaster was carved in ah" 

- iflterestihg way. 

B The electric light was 

switched bh. 
C The furniture was rdiigh but 

well made. 
. D The paintwork was did arid 

.uisattractivc: :. 
E Tlie atmosphere was hot arid 

stuffy. 

20. The warder's^ich j '.iour during the 
vjsit_sho_ws.lhat Jbe_ uas 

A bashfuLjaDd curibux : 

B ilL-manncied and anxious. _ 
C unconcerned and irhpatient 
D embarrassed and .fear_ful 
. E inconsiderate and bored. 

• 21. Which one of the fplipwing a 

tivea best sums up what the author 
feels about *thc spirit of the law' 
(line 54)? It is 
A strict 
B unfair 
C heartless 
D uri^behdirig 
E logical. 



PASSAGE 3 

22. The/ ssage.^ugg^ts^ that J33uch of . 
wJtiai >vriiten_aboul the^ effects of 

: sex^ yiQlence bad language on 
television [s N^_resppnsible^ because 
A there.are definitely no harmful 

effects L 

B it is Nvritien by people oyer 45 
C it is not supp|prted by proper ^ 

research findings 
D there is no need for any 

censorship 
E [he BBC and the ITA are 

always watch fu I. 

23. According to the passage, at the 
rhbitieht television programmes are 
censored^ by 

A the Goverhmeht 

B the television brganlsatidhs 

C trained psychologists 

D parents 

E nobody: 

24. AlUhese staxements about the con- 
duct of the Southern TV investiga- 
tion are true except: 

A thee questions were thought out 

: in advance.: : 

B criticisin_hasedr>Ji specific 

questiom^wasJgnored 

e the_t«earchers were interested 

in people's feelings about; 

television 
D the qucstions were aimed only 

at parents 

* E the investigations >yere carried 

out by means of interviews. 

25. According to the passage, all of the 
fbiiowing statements about tele- 
vision corhpanies are true except 
that they 

" A alneady'^have clear hiles abbiit 
- what they can shbw 
B will make changes in sbme 
-T prbgranimes uiider pf^sure 
C nave studied the problems of 
children and vibleht 

■ -'- prdgrammes 

D turn amy at-present a: sizeable 
proportion jor advertisements 

E consider:extemaii:ensDJship 
of programmes unnecessary. 

26. Which Qne__oLjlhe follo>yi_ng Is 
closest. ia_meaning to 'assess* as 
used inJine^ 13? 

* A Keep up-to-date with 

B Take a sample of 

C Get firm control of 

D Obtain Infonnatipn ab 

E Make less embarrassing 




27. According ta the passage,_the_stated 
purpose of thc^investigaiion carried 
out J5y Sx)uthcrn TV wai to _ 
A tind-out how much 5ex^ and _ 
_ violenceJs shown On television. 
B make, a-responsibie contribution 

to a cpntroyersial issue 

C find put_ i_Qhere_was a cjos^ 

link between television viewing 

and, behaviour 

D discover how parents felt about 

the Lreatment of certain issues 

by television 
E decide whether there was a 

need for strictr'* control over 

television.' 



28. Parents interviewed made all bf the 
foUbwihg objectibhs abbut TV prb- 
grammes except 

A violent scenes might have ah 
- efiect on children 
B children might copy whaj was 
shown 

C some television scenes were 

-- embarrassing 

D some favoarlte programmes 
were shovsfe too late in the 
evening i: i ::: 

E children: might he iliscouraged 
from more ac jive occupations. 

29. TiieSouthem Ji?investigation_indi- 
cates^ alLoflhe fallowing except 

A few people wish a stricter 

contxol J?f progjjammes 

B not all parents worry about 

the effects of television 
C most people pbia:t tb*any 

form of^ censorship 
b ojder^pple ten^ to be more , 

critical of television 
E people's fears are hot confined 

to the effects bf viblehce. 

30. Which twb bf the fbllowihg sen- 
tences b^t summarise the main con« 
elusions bf the passage as a whole? 

(i) There is rib heed to impose 
further censorship dh 
television: 

(ii) The amdiint of violence on 
television is on the increase.: 

(ill) People X)ver 45 yeariold find 
television raoje ofcyectionable. 

(iv) Proof tfmt television is^ a bad 
infiuence oh children is 
Jacking. __ : 

A (i) and (ii) only 

B (i) and fiii) only 

C (i) and Gv3 only 

D (ii) and Oii) only 

E (iii) and (iv) only 
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PAPER V 

Short Story - 1 hour 45 itiinutes 

iNSTRUCtiONS i. The teacher will read the story alpud - follow it in the 
text. 

2. Then read carefaliy page i, headed 'Purposes of Study': 
3: Now read the story again yourself 

4. Tiy td answ^ HEAD the 'Leading Questions' 
on page 1. 

5. WRITE your answers to the 'Assessment Questions' on 
page 2, giving as full answers as you can. 

6. Please note that there is no need to hurry: you have 
plenty of time to think before writing your answers: 

Purposes of Study 

As you study Inexperience have the following purposes iii mind: 

1 . feedings : to fin d but all y o u can abb U t the fee 1 i iigs of tlie gi rl aiid b f 
Andrew, and about the reasons for them. . . . - . 

2. iiight or Wron^ y to judge how far you approve of what they say and 
do, arid hbw far you thirik the author mcarit you to approve. 

3. The Writer's ^m: 

00 to note the writer' , use of the sentng and its relaticviship to the 
characters arid the plot; ^ 

ih) tv> note \;.ues' in the story which suggest the background of the 
characters and their ideas and what is likely to happen as the story 
progresses: 

' (i ) to respond to the shifts of sympathy the author invites from you, 
so tha: you imagine yourseif in the same situation as the characters 
at V\c diiTercnt stages of the story, 



When^ you have read the story again^ answer the : rpjibwirig Auestiohs in your 
head before going^on to the written answers: Finding answers to these prehminaiy 
iujsstibris shcmW help ybii to write belter, rriore corripTete ariswers to the assess- 
ment questions. 

'Leading Questions' (answers not to be written) 

1. Why had Andrew put on his oldest clothes? 

2. Which of the foiiowing words most accurately expresses your feeling for 
the girl at the beginning bf the stbry : 

liking; contempt; sympathy; impatience? 
Justify your choice by thiijking of two points about her. 

3. Whbm wbUld ybu tend tb blarrie for the fact that the Iwb are ribt talking 
to each other in the opening paragraphs? 

4. Pick out 3 or 4 hints in the story that the giri has been plotted against. 
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arid that mUch of the conversation deliberately ignores her. 

5. Why i<lpes Andrew explain about Torn arid Maria 'carefully arid uri- 
naturalJy'? 

6. (a) What becomes clearer \vhen Andrew says, 'Wcii, beyond Hampstead 

then'? : : : :: ^ _ : 

^ (A) What is he doing during the silence while he finishes his drink? 

7. Why is the girl so sibw to beiieve that she is being given the brush-ofT? 

8. Why do you think the author called the story Inexpefience? 

9. What are your feelings about the girl and about Aridrew at the end of 
the story ? 



Assessment Questions 

Ariswcr these qucstibris, as fully as you can, iri writirig: 

Feelings _ 

1. Why are the girl and Andrew not speaking to each other at the 
beginning of the story? ■ - (4) 

2. Trace th & e hariges iri t h e^] rl ' s feel irigs fo r A rid re vy ste]j by step 
through the story. Mark each step by a plirase or sentence quoted 
from the story. _ ' (^) 

3. (a) What afran^emerits have Aridrew arid Coliri rnade before the 

story begins? (2) 
{b) Explain how successful it is and what it telis us about Andrew's 

fee!irig§ towards the girl. (4) 

Right or Wrong 

1. What changes have come over your attitude to the girl between 

the beginning and the end of the story? Explain why: (6) 

?• Wliat feel ings to wa rds A rid re w d b y b ii h av e at the eri d of t he stb ry ? 
Taking into account the probable reasons for what he did, explain 
what led ybu to feel as ybii do. ' (6) 

ne miter's Skiii - - 

1. Mention three things that make us feel the girl is beirig pibtted 
against: (3) 

2. Read these three paragraph^ again: 

(a) Colour flooded into her cheeks. She had forgotten, and she 
was genuinely sorry, for she djd npt want tojbe unkifld tb hjrri. 
EarJi^jr^ at the beginning of the whole thing, shaJiad firmly 
decided n . > tb riiirid abbut his .beirig pbbr (lines 52-54). 

{b) The sudden:iess of the hurt made her gasp arid lose iall bajancc 
for a moment; her expression became ragged and v/ild-- 
Ibokirig: She had known, perhaps long ago, that they were 
arriving at this point. It was too soon, though, far loo soon 
(lines 98-101). . 

(c) This stunned her - for a moment he thought it was because 
she took it as a huge lie. Then hc^saw that she believed hirri 
(lines 139-140): 

Cbrisidering each paragraph separately, explain how the writer: 

(i) helps us to understand more about Tisha and AnclK^ (6) 

(ii) leaves 'clues' suggesting what will happen later in !• c story. (3) 

« 
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Inexperience by Frank Tuohy 

The girl stood with her back to the bar, slightly in everyone's way. A 
pretty girl, she was wearing a dress with a flared skirt that she wore at 
cocktiaiT parties. She swung her foot to and fro, and looked around her at 
the dull groups of raincoated men, whom she took to be commerciai 

5 trayelfers.iHer gj^ces: always ended, with a Httle ^sture: of irritation, asiif 
these people and these places were too boring and typical; then she took 
up her glass, pretending to drink deeply^ She u^uajly^sipged a yei^ littLe, 
sometimes nothing at all. She let the beer touch her mouth and slide back 
i ntb the glass. She wanted to make it last a long tirne. 

iO The young man had on a patched tweed coat and service dress trousers: 
he was about twenty-one and had just come dUt of the Army. He was 
looking down at the top of the bar. Neither of thera spoke. 

Now another young man pushed his way towards the bar, where, after 
exchanging a few words with the landlord, he peered round about him 

!5 until he saw the couple. Their expressions and their way of standing were 
arrahged for the jeneraLpublic, riot for each other. He could guess that 
they had not ieen talking. _: _ 

"HuHo, hullo!' Uie nj^^ was tall arid had a fair haridsbriie 

_ face; his expensive overcoat hung open. 

20 'Qh, Gpm Yau krib'w _ * 

'Of course I know Colin.' The girl turned her face, bright with her 
anxiety to be liked, full on him. 

'Hello there, Fred.' Cblin called girls 'Fred', but he seemed embarrassed. 
'Have I kept ydU waiting?' _ _ _ _ . 

25 'Waiting?' the girl asked, puzzled, for she had not expected to see hirii 
at all. She looke.d at her young man, but he didnotJielp her. . 
' I ' m so rry^ - Co ! iri said s ud den ly i: ro usiri g hi rriselF. ' Wh at aveyp u d rinking?^ 
'Gh, thank you,' the^irl said, 'bat I'm all right with thi*- one.' The young 
riiari was fre\vn i at Go Ijri acros^ her head . _ ^_ : _ _ 
30 'r see.' He caught the landlord's eye, and bought drinks for himself and 
his friend. 'What Ve ybU been dbirig?' s . 

'We went to the Gurzon,' the girl said helpfully. 'It was really awfully 
good, wasn't it Andrew?' 

'You said you liked ii,' her young man answered. 
35 'WeU, you liked it too, you know you did.' 

'Me?' He made ah incredulous voice. 'I thought it itari_k.'_i i „i 
Then why did you take mc to it, if you thought it was so awful?' she 
asked crossly^: i i : i _ _ iz 

To interrupt, Colin said: 'What was this film, Fred? You haven't said 

40 what it was yet.' ^ _ 

She told him the name of the fiini. 'Have you seen it?' 
>'Yes.' 

'Well, you liked it, surely?' 

Colin looked across at Andrew, laughing, 'i must say I thought it stank, 

45 too.' : : : ' zl 

The girl made a curious trapped niovement of her head, 'ft was jolly 
good,' she sajdj J^linkin^. 'I thbugh^t ]| w^ 

stitching of one of her gloves and started humming a little tune to herself^ 
then she carne back at A^ 'It's ybUr turn to buy a drink/ ' 

50 He made a hopeless face at her. 'Tisha, you know perfectly well I haven't 
any more money. I told you.' 

Colour flooded into her cheeks. She had forgotten, and she wa^genMnely 
sorry, for she did not want-to be unkind to him. Earlier, at the beginning 
of the whole thirig, she had firriily decided riot to riiirid about his be^rigpobr. 



55 'Give me a cigarette then,' she said qliite softly. 'Look, there's a table 
free; why don't we sit down?' 
^ But s^e f?It their resistance as she said it, arid again she had the feeling 

that she had said che wrong thing. They remained standing, and soon 
- afterwards the two men began a convefsation about pebple she didn't know. 
60 Witlibu^L Mteriirig to them she smoked her cigarette in little puffs, with a 
siight frown. _ : ^ 

L 'T^M MariaJ CoIJn was saying, 'They're in a pretty bad way, I've 
been up there aii afternoon and I did say we'd go back there; I don't think 
they should be left alOT 
65 The girl looked from one to the other; her eyes half-closed; as if trying 
in a sophisticated v^^^ 
'What sort of thing was happening?' 

'The usual things. They'll probably be turned but bf the studib.' 
'I see.' _ 

70 The^rl watched Andrew cafefiilly whenever he talked. She was anxibus 
about him. It had be^n Sunday afternoon and she was aware that all had 
not gone very well. She had dressed up, but he had appeared in his oldest 
clothes. He had not sjb^^ increasingly obvious to her 

that he reserved his humour for state occasions. One always realised things 

^5 so niuch mbre deeply a_fter the cinema: corning du Gurzon this 

afternoon into the coid streets of early summer; she knew thathe no longer 
^ried to interest her. Withbut riieariirig tb, she sighed; but neither bf them 
noticed. 

Colin said rather Ibudly ; 'Yes, I tbld them^ 
80 : She looked up at him. She hadn't known they were going to meet Colin 
this evening, and there was something nbt fight abbut his being there. 
Nevertheless^ his presence might make th 

'We must do-that, then,' Andrew said. He turned tb the girl and began 
explaining ca^refujly and unnaturally. 'Tom and Maria - these friends of 
85 ours - yx)u've heard me speak of them? Well, Tom's wife been writing to 
him again arid they're both very depressed.' 
'I see.' _ 

'Cbliri's beeri but then^ this aftero^^ ^ he brily cariie back again to see 
us._ He said we'd go up there tonight.' 

coufse, why not?' Since this fitted in with her idea bf bcirig^ among 
other pcfople this evening, she let herself agree with what he was saying. 
'Well, let's go there then." 
'Butj Tjsha, it'll be a great bore.' His voice was quieter. 
- 'tdont see why; they're friends of yours, aren't they?' 
95 'Bu_t Jisha ^ 

'When do we start?' 

'Not you, Tiaha.' _ ~z ii i-n-i 

The suddenneS of the hurt made her gasp and lose all balance for a 
momen t ; h ef expressi bri became ragged and wi Id-Iqbkirig. Sfi e had kri b wri , 
100 perhaps long ago, that they were arriving at this point. It was too soon, 
though, far too soon. 

'Well - r - rcally!' 

'But: Tislia,' he was going on saymg, 'you don't_know these people.' 
:_/J i^pvvl don-t.iBut am I so awful that I can't be shown to them? Yd 
105 think i was a pariah* or something: Look, I want to meet yx)ur friends. I 
^^"Pw you don't tW tp, hut Ldb.' Suddenly it was impdrtant for 

her to fight, though she hated herself for it. 

*10t to g^^ into a flap. Colin has bnly just tbld me I have to go arid 
'iee these people. I don't want to go, it's a long way and it'll probably be 

♦pariah : an outcast. 



no very dull.' 

•Where is it?' 

'Hampstead.' 

'Hampstead's not very fan' 

*WeII, beyond Hampstead then. Anyway, we're not going to have a gay 
1 !5 timej a smart time It wouldn't be at all interesting for you.' 

*Hdw do you know?' she asked him. 'Ybu think just becauise I don't 
belcmg to your lot and I still live at home, that I don't know anything at all.' 
•It isn't that - ' 

_jlf you mean tH^^ won't like me, why don't you say so? Is there sorne- 

120 thing I've donC-wrong?' _ 

'No? really, Tisha, stop this.- He^gave a groan of tiredness. *Why do you 
always put everything on to yourself?' ^ 

'Because I believe that's what it's all about. I don't think you're going 
anywhere. I think you're just trying to get rid of me.' 
125 - Tisha, ydU heard Colin tejl me, didii't ybii? Ybli heard him say that 
Tom and Maria wanted to see me ?'^ 

She gave a little hard laugh, but' she could not quite turn and c^Il the 
pther young man a lian -It all sounds yeiy suspicious to me^' she said, 
__ *Why don't you let him ga by himself?' She jerked her head mdely at 
130 Colin, who v^l looking al himseL^^ 

of the bar: He turned, bat his glance . seemed to slide ^cwsy from her face. 
Tllfy asked nle td^g^^ Andrew said. 'I can't let them down.' 
*What about me? You had a previous appointment to take me out:' An 
^'^^ '^V^^^o her, \l 

135 any money, I'll go home and borrow some from my brother.' ^ . . . 

There was a silence while Andrew finished his drink. She watched him 
ruriously, twisting her gloves in her hands. He put his glass down arid said: 
'I didn't want^to tell you this, Tisha. Tom tried to commit suicide last week.' 
iThis stunned her - for a moiherit he thought it was because she took it 
140 as a huge lie. Then he saw that she believed him. ^ 

L-phi_I see.'. Her voice carn_e_ thin_ahd strained^ 'Well, I don't see what 
you can do about it. He's not going to do it again, is he? He probably - 
^id i t to sh b w off . ' 
'How dare you say a. thing like that?' ; 
145 'No; I'm sorry 1 - ' 

'What right have you to suggest such things - yoi'., just a spoiled little 
girl who talks too much.' 

She looked at him and for a moment she was horrified with herself. She 
-- was sure now that she would not see hjm again, but she knew that she 

1 50 wo uld go cm lo ving him fo r is ome^ time :y e t . : : 

'We ought to go,' Colin said, looking ^t his watch. 
'I don't kri o w where I - m: go ihg to gb^' she * said pat hetica lly, * al I d resspd 
up like this, I've said I'll be out to dinner. Please couldn't you take mc 
with ybU - I'll wait bUtside.' 
155 Aridrew; avoiding going near her, almost shouted: tell you, Tisha, it's 
notpossible.' 

Their (^es met, wounded, angry and meaningless, fhcy stared at each 
other for some time. j : 

-Oil, very weIL th^cn.j3j^^ _ : . : : ^ 

166 She went towards tlie door, her head held up, tottering a little on black 
CO u rt sh be s wi th to b-h igh heels . They watched h e r uh ti 1 sh c had gon c . 

Andrew broke into a sniggering mock-dance. 

'Oh Gbd! Daiftn, dairin, damn! Mbre drink, quickly. You know, ybu 
saved mc, being there. I thought I was going to give in." 
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165 'Let's go out somewhere and get drunk, shall we?* 

-Yes - look out!' : :. : : *. 

She was standing between them, her face white and shocked: 
Tve leH one of rhy gloves behind/ she said. - 

_ _ _ _ " ^ 

{ Reprinted by kind permission of A, D, Peters <5c Co, Lfdj 



SCRE/SeEEB ^eSiTERlON TEST' 
PAPER V - SHORT STORY 
Guidance for Markers 

1. in marking the answers to Ihexpericnce ignore the mark^^ 

questions. You should j^ive each script mi IMPRESSION MARK out of 
20. Be prepared to use the full length of the scale of rri^rk^. jjf^ouia^ 
Used to giving marks for 'Reading' questions out of 16^ it does no harnt 
to grade answers that way first and then translate them into scores but of 
20.) 

2. Your assessment of the quality of answers should take the following general 
points into account: 

1 . The pupil's awareness of Tisha's feelings for Andrew and of his towards 
: hot = 'FEELINGS', 

2. The pupil's judgment of both tiisha and Andrew and the degr^e^ to 
which the pupil has been influenced in Uiis judgrnent by all the infor- 
mation the writer gives us about the two characters= WG^^r OR 
WRONGr. 

3. The pupil's iwarene^s of 'clues' in the story relating to (a) the likely 
butcome of events (b) the characters and social background of Tisha 
?ind Andrew= 'WRITER'S SKILL ^ 

Notes: 

(a) Relevance is to the generci pomt rather than to specific q^uestioris. 

E.g., -any accurate bbscfvatibns about Tisha's andAndrew'5 feel- 
ings for each other made in answer to any of TEELJNQSi Cju^^^ 
2 and 3' shbujd be given credit; but answers: whicfau discuss xHiLy. the 
characters' feelings for each other under- BRIGHT OR WRONG' 
should be regarded as irrelevant and receive no credit. 

Remember that an oVerait impressron of tire quality of the pupil's response 

/o /& 5/00^ is what is wanted. 

(b) Unlimited time was allowed for this test. It is reasonable lo expect 
an attempt at alt three sections from any pupil obtatnmg a pass mark. 

(c) The list of possible .points Jiveri ibelow is riot exhaustive, but is prb- 
haSly much longer than we can reasonably expect of 0-grade pupils. 
Roughly speaking, brie rilight divide answers into the following three 
categories: 

1. GOOD ANSWERS : 

FEEhrNCS: Show awareness of Tisha's uncertainty AND 
arixiety AND affection AND resentment AND (probable) 
bitterness at the end. * 

Show awareness of Andrew's desire to be rid of her AND his 
feeling that she is tiresorile arid clinging, spoiled and shallow: 
RIGHT OR WRONG? Show a change in attitude towards the 
girl, probably from a slightly critical or cmnmrptaous niiiiude 
(She's 'green', slow on the uptake^ uhdignjfied, ^^^s^^^ 
sympathy because she is vulnerable, the victim of a conspiracy 
and is humiliated. 

Show a critical reaction to Andrew (his behaviour cannot be 
171 ^ ^ 
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fully appro vedj)^ but ^t^ some awareness of his pro b- 

lem:^ he 5 inexperienced too; she's hard to throw off, yet she's 
not *bn his wavelength' ... 

In both cases there will be a /w/V statement of reasons for'the judgments 
made. 

WRITER'S SKILL: Show awareness of 3 different clues sug- 
gesting conspiracy. 

Show understanding of ///rf-e orybwr background and character 
points made by the paragraphs quoted, and of two or three 
indications given in them by the writer of the way events ard 
tending. 

ri; AVERAGE ANSWERS: 

FEELINGS: Answers will make the most obvious points, but 
omitj» e.g.^Tisha's unce anjd her later resentment, arid show 

no awareness of ir/jj^ Andrew wants rid of her. 

RIGHT OR WRONCj? Answers will show some awareness of 
how the writer has guided the readers' re^^ to the characters, 
bat may indicate: that only the most obvious clues have-been 
rioted : ^i^g-x initial dijlike leadirig^^ to final lyinpathy for Tisha, 
simple criticism of Andrew^ with adequate reasons foi the judg- 
ri^^rit, but lackirig^ that riiariy aspects of their 

personalities and circumstances are hinted at. 

WRITER'S SKILL: Show awareness of two or three 'clues' 
about the cbrispiracy, but possibly riot the most sigriificarit dries. 

Show understanding of at least one background or character 
point made by each of the three paragraplls quoted, and of at 
least brie *clue' about later events contained iri them. 

in. POOR ANSWERS: 

Will show a failure Ihrougliou^ with the ijues- 

tions set and appreciation of only the surface events of the stoo^ 
lacking any but the most rudiriientary awareness o^^ 
and social situation and of the different personalities involved. 

Iricxpericricc : List of possible points for each question 

Feelings: 

^- ^*L^rew is fed up w^^^^^^ 

She wasn't at ease, didn't know what to say. 

Strained relationship. 

She is a rather shallow person. 

They have little in common: 

2: Puzzled 
? yh9<2rtairi 11 ^1 : _ _ 
Vaguely worried about losing him 
WMts his atteri tiori 
Some affection (love) 
Suspicious 
Hurt 
Resentful 
Angry 
Shocked 

(Probably) bitfer at erid. 
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^ J J(st pr^^^^ l^ere if it dbes indicate that the pupil 

!iHs noted chiir^ci Iw *v:r feeilngf. towards Ahdrew, 

3 (a) TJ',at CcUii would come be a gooseberiy. 

Th.. t CO'H \\ ouid spin about Torn arid W aria. 
^'.> ::ct rid ^ f Tisha so tl ..t they would go and get drunk. 
(h) ' 'aoi! - I occessfal, it was more embarrassing than Andrew expected 
• _ I * ^ ' pr it wasii t - she was susp^^ 

A: 7 : fed up with her - didn't want to be direct - afraid she'd 
oy^rcoiTK Jiim if he didri't have help. Lacking cbixrage - lacking real 
feeling ^'or her. 

Righf or Wrong?: 

1. (i) sympathetic ? - or appe^ar^s a-bit snooty, affected, sha^ 
(ii) f'eel she's 'green' Xfilni diaiogue) 

■ ^ d rag' (hangs on to A rid re w) 
in need of support 

sibw in the Uptake (re brush-off) ^ 

undignified (has to fight 
all dressed up 
said she'd be pilt to dinner 
Tlease take me') 

(iii) sympathetic again -^she's a victim of cdnspiracy. - 

she was vulnerable. ^ 
she is humiliated. 

2 . Di Hike, especially: For 'sriiggeririg mock-dance She was_ pro ba bjyi hard 
to throw off-- sympathy for him ia this. Dislike his dishonesty, trickery. 
Peeltjia t he had mo re expe rience th an Jhe -_sh bu id ' ye kn 6 wn be tte r how 
to end the affair. Or just 'He didn't know how to do it any other way 
(arid rieeded Coliri's support).' 



yVrit^r's Skill: 

1. Colin^s arrival is pre-arranged, but Tisha doesn't know. 
And re w f ^Q^yns b ve r he r headi at Col in 

ColiiL clearly 'on Andrew's side' about film. 

She Telt thejr resistanc^^^^ 57) (felt excluded^., _ 

'Golin said rather loudly', ' . : . go back . . . bring you (Andrew):' (1 79) 

Whole episGt^le of P rejected 'H ampstead ' y i si t . _ 

Andrew turns Tisha's natural reaction against herself (1 1 139-147 - the 

'suicide'). 

The men's merriment. (I I 162-164.) 

2. (i) (a) Social difference Hi dic^^ 

Also she does feel genuinely for him - yet she has io firmiy 
cfecTde riot to mirid about hisjjeirig poor. 

( b ) S he: refu ses to be jbr u shed_ off beca li se s he Mows iu he r hea rt 
that this is the feared end of the affair: She is romantic/unrealistic 
and yulnerabje. Andrew, however clurilsily, had tried to avoid 
causing sudden hurt: 

(c) (]) So brutal a reality sorts ill with her romantic outlook* 

(2) SKe is 'green' riot to see through the cock-arid-bull story 
• or the reader should rtalisc v'<iat the point is that it is a lie. 




(iii) Andrew is jockeying fofii better position; He doesa'twan 
to be caught out; doesn't want to be direct y/ith Tisha;/s 
iiiore coricerried with successfully throwing her than with 
the truth: / 

(a) Their social discrepancy is already pulling their • : -md will 
probably continue to do so / 

(b) It is *the point of no retuir . 

(c) Andrew is on a winning tack he can :4^>:> ;^^nage' ijsha 
or - it is this lie which iead^ to her huinil-ation. 
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Appendix 5 
Trial Marking 

Appendix 6 

^Visiting Moderator* ISchenie 
Appendix 7 

Two Examples of School Ranges, Moderator' Ranges and O-grade 
Ranges Compared 

These Appendices are not included -in-this pubfication but are available as 
separate items, on request, from the SCE Examination Board. 
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Appendix 8 



Effect of Scaling in Tw5 Schools with 'Satisfactory' Internal 
Assessments 



I Characteristics of School 24 and t^^^ 

The cdrrelatidiis between the Fdlid and the other assessments at this sx;hbc)] 
arc very high (FoliD/Criterion Test: -89, Folio/O-grade; -84) but one would be 
wrong to assume that this means^ a close sirailarity of the results for each and 
every pupil. In the first place the 'tail' of poor pupils makes a strong contrU 
bution to the correlation, and if those pupils who are Judged as range 13 or 14 
by : t}ie_^_chopI are ex^^^ (since in other schools they 

would not be presented) together withJhose^ho obtain an 'F' in O-grade the 
cbrrelatioris are sub^ ;7i5, Fblio/O-grade: 

•'':). Secondly, a gr^h of the Folio assesrment compared to the Criterion Test 
( see Figure A 8 • 1 ) dbes rid^hbw a uii ifo nn C51 rye^ and tb e _c_h a racteris t ics of 
this picture can be made clej^y identifying the class to which each pupil belongs: 

Although the school was asked to produce an all-through assessment, they 
have not altogether succeeded in so doing. One class in Plartieular {Class_2J 
. stands put as^a) not succeeding in using a ft)mparable standard to ^he remainder 
' of the school a^s well as (b) failing to produce a satisfactory ratik order. In 
contrasti^ another clpss^^(^ 3j was assessed with remarkable accuracy - way 
beyond that of the O-grade - though on :a standard matched to the Criterion 
T^st father than t^' the- less seyere O-grade. In consequence, when it was attempted 
to standardise the Folio Marks for the entire schooi a variety of eflfects resulted. 
Since Class 2 was geri to a lower mean 

mark' overall and an increase in the severity of marking of Ciass 3 -^a^eryomfair 
effect if Jt were to influence t^ to thpse^upils. Fbr Q 

itself the rnarks remained too high after scaling, but the major problem resulted 
from the initially poor rank ordering oi* these pupils. Rank brde ring bf this « 
class was certainly not easy, sincf^ they are concentrated in bands B and C; the 
result of the school having made discriminations which do lidt correspond tb 
{hose of the Critenp^n is that puplils of apparently SBir-iihr com- 

petence obtain different marks before scaling and these differences are grdssly 
exaggerated by the scaling procedu^ of jthjs class would obtain, 

scaled marks which would mSke them seem more dissimilar than fiiey really 
are. The assessrnen t of the . top c lass, CI ass 1 , was iieitherp bp r^n especi al ly 
good on rank ordering - it can be taken as representative of the majority of 
assessments in this respect. Class 1, however, was most successfully matcK&djio 
th^ uriscaled P-grade marks on standard. Where standard is correct but agree- 
m' - ' n ranking is not^ close, no benefit js achieved by any scaling prbcedure. 
An i>yi_bus irnprbvement m^ class, from combining 

the . olio and O-grade assessments, without any statistical adjustmeiits. 

Thus this one school is a microcdsm in which we cail find excellent ranking 
on the ^yr<Mlg_ standard (Class :3)i poor ranking combined with a wrong standard 
(Class 5), and average ranking on the correct standard (Class 1) - a vety useful' 
example. Fulrtheri frbm^ it appears that these 

differences in assessment among the classes are ahnost ^ttrely accounted for 
by the Folio interpretatipji secy (See Chapter Viy_The folio cbmpdsition 
marks for Class 2 are slightly high, but the main disagreement of standard is 
on interpretation. 

l^igure A8.2 shbv^^^ and 3: the. 

'cloud' of crosses representing class 2 has been spread out so that some pupils 
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are further away from the Mdeal' agreement of istandafd than before and members 
of class three are even more severely dealt with than previously,^^ 

SchooJ 22 presents a quite different picture. The correlation coefficients- for this 
school are similar to thbse^^^ 

•79, but the agreement on ranking was maintained throughout the school, ft 
can be seert from Figure A8.3 that the Fblib ranking in schbbl 22 was quite 
good by comparison with the Criterion Test rank order. There was more scatter 
when Folio marks were plotted against O-grade results (Figure A8.4); but 
ranking is consistent, while there is a clear severity on all pupils combined with 
an unwrfilingness to give the- best pujjils high marks. In these circumstances, the 
sccyirij j^rpcedure wa_s_ h^^^ mark arid pushing the best 

pupils up to the top of the mark scale. (See Figure A8:5.) 



ft is, however, also obvious from Figure A8.5 that the stretching effect of the 
scaling procedure exaggerates the discrepancy v^^hen there is lack of agreement 
on ranking. 



Ml 



i7n 



FfGURE A 8.1 Kolio in School 24 before scaling against Critcridri Test 




FIGURE A 8.2 EfTects of scaling on classes in SchbbI 24 




FIGURE A 8.3 Ravv Fdlib against Critefibri Test - Sbhdbl 22 
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FIGURE A 8.4 Raw Fdlid against O-grade - School 22 
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FIGURE A 8.5 Scaled Folio against O-grade - School 22 
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2<^pipeiidix 9 

Bistributibhs of Marks for Eriteriori Test Markers 



This Appendix is not included in this publication bat is available as a separate 
Iten., on request, ffOrti the SCE Examination Board. 
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Appendix 10 Distributions of Raw Marks: 
Folio and Ordinary grade 
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Appendix ii 



Means and Standard Deviations (JRafv Marks): O-grade, Folio and 
Criterion Test (possible total marks in each case^^iOO) 



O'grade (ra^) 



Folio {raw) 



Criterion Test 









._ _ ... 
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School ' 
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MEAN 


s.b. 
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. S.D. 












J I 
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9-87 


57-28 


10-97 


51-30 


8-80 


i3 


1 63 


53-: 


12-91 


.55-04 
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45-30 
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i4 
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12-04 


54-76 
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11-30 


is 


154 
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55-23 
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21 
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9-53 
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10-35 
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11-50 


22 
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23 


81 
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13-10 
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13-40 


24 


70 
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12-03 


52-27 


13-59 


50-20 
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.32 
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11-38 
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47-30 


11-20 


33 


45 ' 


54-38 


9-il 


55-51 


8-51 
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9-30 


.15 
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59-16 


i i -53 - ; 


57-93 


10-78 i 
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12-20 
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Cro^taMation of Correlation Coefficients for all Assessment 
Elements in the Project (n=1529j 
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